{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Softmax 练习\n",
    "\n",
    "\n",
    "*完成并连同作业上交你完整的工作表(包括其输出及工作表外的任何支持代码)，更多详情请参见课程网站上的[作业页面](http://vision.stanford.edu/teaching/cs231n/assignments.html)。*\n",
    "\n",
    "这个练习类似于SVM练习,你会:\n",
    "- 为Softmax分类器实现一个完全向量化的**损耗函数**\n",
    "- 实现其**解析梯度**的全向量化表达式\n",
    "- **检查你的实现**与数值梯度\n",
    "- 使用验证集**调整学习率和正则化**强度\n",
    "- 用**SGD优化**损耗函数\n",
    "- **可视化**最后学习的重量\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import numpy as np\n",
    "from cs231n.data_utils import load_CIFAR10\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (12.0, 9.0) # set default size of plots\n",
    "plt.rcParams['image.interpolation'] = 'nearest'\n",
    "plt.rcParams['image.cmap'] = 'gray'\n",
    "\n",
    "# for auto-reloading extenrnal modules\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train data shape:  (49000, 3073)\n",
      "Train labels shape:  (49000,)\n",
      "Validation data shape:  (1000, 3073)\n",
      "Validation labels shape:  (1000,)\n",
      "Test data shape:  (1000, 3073)\n",
      "Test labels shape:  (1000,)\n",
      "dev data shape:  (500, 3073)\n",
      "dev labels shape:  (500,)\n"
     ]
    }
   ],
   "source": [
    "def get_CIFAR10_data(num_training=49000, num_validation=1000, num_test=1000, num_dev=500):\n",
    "  \"\"\"\n",
    "  Load the CIFAR-10 dataset from disk and perform preprocessing to prepare\n",
    "  it for the linear classifier. These are the same steps as we used for the\n",
    "  SVM, but condensed to a single function.  \n",
    "  \"\"\"\n",
    "  # Load the raw CIFAR-10 data\n",
    "  cifar10_dir = 'cs231n/datasets/cifar-10-batches-py'\n",
    "  X_train, y_train, X_test, y_test = load_CIFAR10(cifar10_dir)\n",
    "  \n",
    "  # subsample the data\n",
    "  mask = range(num_training, num_training + num_validation)\n",
    "  X_val = X_train[mask]\n",
    "  y_val = y_train[mask]\n",
    "  mask = range(num_training)\n",
    "  X_train = X_train[mask]\n",
    "  y_train = y_train[mask]\n",
    "  mask = range(num_test)\n",
    "  X_test = X_test[mask]\n",
    "  y_test = y_test[mask]\n",
    "  mask = np.random.choice(num_training, num_dev, replace=False)\n",
    "  X_dev = X_train[mask]\n",
    "  y_dev = y_train[mask]\n",
    "  \n",
    "  # Preprocessing: reshape the image data into rows\n",
    "  X_train = np.reshape(X_train, (X_train.shape[0], -1))\n",
    "  X_val = np.reshape(X_val, (X_val.shape[0], -1))\n",
    "  X_test = np.reshape(X_test, (X_test.shape[0], -1))\n",
    "  X_dev = np.reshape(X_dev, (X_dev.shape[0], -1))\n",
    "  \n",
    "  # Normalize the data: subtract the mean image\n",
    "  mean_image = np.mean(X_train, axis = 0)\n",
    "  X_train -= mean_image\n",
    "  X_val -= mean_image\n",
    "  X_test -= mean_image\n",
    "  X_dev -= mean_image\n",
    "  \n",
    "  # add bias dimension and transform into columns\n",
    "  X_train = np.hstack([X_train, np.ones((X_train.shape[0], 1))])\n",
    "  X_val = np.hstack([X_val, np.ones((X_val.shape[0], 1))])\n",
    "  X_test = np.hstack([X_test, np.ones((X_test.shape[0], 1))])\n",
    "  X_dev = np.hstack([X_dev, np.ones((X_dev.shape[0], 1))])\n",
    "  \n",
    "  return X_train, y_train, X_val, y_val, X_test, y_test, X_dev, y_dev\n",
    "\n",
    "\n",
    "# Invoke the above function to get our data.\n",
    "X_train, y_train, X_val, y_val, X_test, y_test, X_dev, y_dev = get_CIFAR10_data()\n",
    "print ('Train data shape: ', X_train.shape)\n",
    "print ('Train labels shape: ', y_train.shape)\n",
    "print ('Validation data shape: ', X_val.shape)\n",
    "print ('Validation labels shape: ', y_val.shape)\n",
    "print ('Test data shape: ', X_test.shape)\n",
    "print ('Test labels shape: ', y_test.shape)\n",
    "print ('dev data shape: ', X_dev.shape)\n",
    "print ('dev labels shape: ', y_dev.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Softmax 分类器\n",
    "\n",
    "本节的代码将全部写在**cs231n/classifier /softmax.py**中\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss: 2.352635\n",
      "sanity check: 2.302585\n"
     ]
    }
   ],
   "source": [
    "# First implement the naive softmax loss function with nested loops.\n",
    "# Open the file cs231n/classifiers/softmax.py and implement the\n",
    "# softmax_loss_naive function.\n",
    "\n",
    "from cs231n.classifiers.softmax import softmax_loss_naive\n",
    "import time\n",
    "\n",
    "# Generate a random softmax weight matrix and use it to compute the loss.\n",
    "W = np.random.randn(3073, 10) * 0.0001\n",
    "loss, grad = softmax_loss_naive(W, X_dev, y_dev, 0.0)\n",
    "\n",
    "# As a rough sanity check, our loss should be something close to -log(0.1).\n",
    "print ('loss: %f' % loss)\n",
    "print ('sanity check: %f' % (-np.log(0.1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inline Question 1:\n",
    "为什么我们期望损失值接近-log(0.1)?请简要解释\n",
    "\n",
    "**Your answer:** *Fill this in*\n",
    "\n",
    "- 因为W随机初始化（有10个类别），所以每个类计算的得分是相同的，经过SoftMax之后的概率是一样的，因此每一个类的概率都是0.1，求得交叉熵也就是-log(0.1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "numerical: -1.509718 analytic: -1.509718, relative error: 4.266408e-09\n",
      "numerical: 0.862750 analytic: 0.862750, relative error: 4.505671e-09\n",
      "numerical: 1.263459 analytic: 1.263459, relative error: 5.402075e-08\n",
      "numerical: -3.421720 analytic: -3.421720, relative error: 1.897553e-09\n",
      "numerical: -0.557343 analytic: -0.557343, relative error: 3.774957e-08\n",
      "numerical: 1.287508 analytic: 1.287508, relative error: 6.741293e-08\n",
      "numerical: 1.336710 analytic: 1.336710, relative error: 5.571692e-08\n",
      "numerical: -1.449432 analytic: -1.449432, relative error: 1.354936e-08\n",
      "numerical: -1.259372 analytic: -1.259372, relative error: 1.024200e-08\n",
      "numerical: 1.735493 analytic: 1.735493, relative error: 4.379149e-08\n",
      "numerical: -0.386872 analytic: -0.386872, relative error: 6.325432e-08\n",
      "numerical: -4.894356 analytic: -4.894356, relative error: 1.351140e-08\n",
      "numerical: 1.749314 analytic: 1.749314, relative error: 5.216199e-08\n",
      "numerical: 0.683288 analytic: 0.683288, relative error: 1.809120e-07\n",
      "numerical: 1.983511 analytic: 1.983511, relative error: 1.319008e-08\n",
      "numerical: -2.393032 analytic: -2.393032, relative error: 2.735724e-09\n",
      "numerical: 3.879182 analytic: 3.879182, relative error: 7.987180e-10\n",
      "numerical: -2.228998 analytic: -2.228998, relative error: 3.837043e-09\n",
      "numerical: -4.680740 analytic: -4.680740, relative error: 1.332305e-08\n",
      "numerical: 1.645220 analytic: 1.645220, relative error: 2.225834e-08\n"
     ]
    }
   ],
   "source": [
    "# Complete the implementation of softmax_loss_naive and implement a (naive)\n",
    "# version of the gradient that uses nested loops.\n",
    "loss, grad = softmax_loss_naive(W, X_dev, y_dev, 0.0)\n",
    "\n",
    "# As we did for the SVM, use numeric gradient checking as a debugging tool.\n",
    "# The numeric gradient should be close to the analytic gradient.\n",
    "from cs231n.gradient_check import grad_check_sparse\n",
    "f = lambda w: softmax_loss_naive(w, X_dev, y_dev, 0.0)[0]\n",
    "grad_numerical = grad_check_sparse(f, W, grad, 10)\n",
    "\n",
    "# similar to SVM case, do another gradient check with regularization\n",
    "loss, grad = softmax_loss_naive(W, X_dev, y_dev, 1e2)\n",
    "f = lambda w: softmax_loss_naive(w, X_dev, y_dev, 1e2)[0]\n",
    "grad_numerical = grad_check_sparse(f, W, grad, 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "naive loss: 2.352635e+00 computed in 0.133642s\n",
      "vectorized loss: 2.352635e+00 computed in 0.003989s\n",
      "Loss difference: 0.000000\n",
      "Gradient difference: 0.000000\n"
     ]
    }
   ],
   "source": [
    "# Now that we have a naive implementation of the softmax loss function and its gradient,\n",
    "# implement a vectorized version in softmax_loss_vectorized.\n",
    "# The two versions should compute the same results, but the vectorized version should be\n",
    "# much faster.\n",
    "tic = time.time()\n",
    "loss_naive, grad_naive = softmax_loss_naive(W, X_dev, y_dev, 0.00001)\n",
    "toc = time.time()\n",
    "print ('naive loss: %e computed in %fs' % (loss_naive, toc - tic))\n",
    "\n",
    "from cs231n.classifiers.softmax import softmax_loss_vectorized\n",
    "tic = time.time()\n",
    "loss_vectorized, grad_vectorized = softmax_loss_vectorized(W, X_dev, y_dev, 0.00001)\n",
    "toc = time.time()\n",
    "print ('vectorized loss: %e computed in %fs' % (loss_vectorized, toc - tic))\n",
    "\n",
    "# As we did for the SVM, we use the Frobenius norm to compare the two versions\n",
    "# of the gradient.\n",
    "grad_difference = np.linalg.norm(grad_naive - grad_vectorized, ord='fro')\n",
    "print ('Loss difference: %f' % np.abs(loss_naive - loss_vectorized))\n",
    "print ('Gradient difference: %f' % grad_difference)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "iteration 0 / 5000: loss 778.474061\n",
      "iteration 100 / 5000: loss 285.719183\n",
      "iteration 200 / 5000: loss 105.871889\n",
      "iteration 300 / 5000: loss 40.077869\n",
      "iteration 400 / 5000: loss 16.006308\n",
      "iteration 500 / 5000: loss 7.166896\n",
      "iteration 600 / 5000: loss 4.015165\n",
      "iteration 700 / 5000: loss 2.801743\n",
      "iteration 800 / 5000: loss 2.322427\n",
      "iteration 900 / 5000: loss 2.212780\n",
      "iteration 1000 / 5000: loss 2.095009\n",
      "iteration 1100 / 5000: loss 2.072824\n",
      "iteration 1200 / 5000: loss 2.132577\n",
      "iteration 1300 / 5000: loss 2.068940\n",
      "iteration 1400 / 5000: loss 2.086209\n",
      "iteration 1500 / 5000: loss 2.059195\n",
      "iteration 1600 / 5000: loss 2.049228\n",
      "iteration 1700 / 5000: loss 2.121531\n",
      "iteration 1800 / 5000: loss 2.098930\n",
      "iteration 1900 / 5000: loss 2.063414\n",
      "iteration 2000 / 5000: loss 2.121234\n",
      "iteration 2100 / 5000: loss 2.113409\n",
      "iteration 2200 / 5000: loss 2.070880\n",
      "iteration 2300 / 5000: loss 2.060384\n",
      "iteration 2400 / 5000: loss 2.073054\n",
      "iteration 2500 / 5000: loss 2.133219\n",
      "iteration 2600 / 5000: loss 2.092272\n",
      "iteration 2700 / 5000: loss 2.085760\n",
      "iteration 2800 / 5000: loss 2.143933\n",
      "iteration 2900 / 5000: loss 2.147190\n",
      "iteration 3000 / 5000: loss 2.074040\n",
      "iteration 3100 / 5000: loss 2.126094\n",
      "iteration 3200 / 5000: loss 2.098448\n",
      "iteration 3300 / 5000: loss 2.045897\n",
      "iteration 3400 / 5000: loss 2.050875\n",
      "iteration 3500 / 5000: loss 2.092273\n",
      "iteration 3600 / 5000: loss 2.121242\n",
      "iteration 3700 / 5000: loss 2.096166\n",
      "iteration 3800 / 5000: loss 2.112260\n",
      "iteration 3900 / 5000: loss 2.028006\n",
      "iteration 4000 / 5000: loss 2.057070\n",
      "iteration 4100 / 5000: loss 2.122445\n",
      "iteration 4200 / 5000: loss 2.100609\n",
      "iteration 4300 / 5000: loss 2.067597\n",
      "iteration 4400 / 5000: loss 1.984524\n",
      "iteration 4500 / 5000: loss 2.101154\n",
      "iteration 4600 / 5000: loss 2.056764\n",
      "iteration 4700 / 5000: loss 2.084586\n",
      "iteration 4800 / 5000: loss 2.048077\n",
      "iteration 4900 / 5000: loss 2.059211\n",
      "iteration 0 / 5000: loss 15454.904926\n",
      "iteration 100 / 5000: loss 2.264855\n",
      "iteration 200 / 5000: loss 2.270472\n",
      "iteration 300 / 5000: loss 2.266886\n",
      "iteration 400 / 5000: loss 2.285547\n",
      "iteration 500 / 5000: loss 2.261620\n",
      "iteration 600 / 5000: loss 2.259171\n",
      "iteration 700 / 5000: loss 2.282443\n",
      "iteration 800 / 5000: loss 2.283134\n",
      "iteration 900 / 5000: loss 2.256323\n",
      "iteration 1000 / 5000: loss 2.271702\n",
      "iteration 1100 / 5000: loss 2.266864\n",
      "iteration 1200 / 5000: loss 2.269261\n",
      "iteration 1300 / 5000: loss 2.268422\n",
      "iteration 1400 / 5000: loss 2.260687\n",
      "iteration 1500 / 5000: loss 2.273920\n",
      "iteration 1600 / 5000: loss 2.268779\n",
      "iteration 1700 / 5000: loss 2.261868\n",
      "iteration 1800 / 5000: loss 2.267234\n",
      "iteration 1900 / 5000: loss 2.271555\n",
      "iteration 2000 / 5000: loss 2.259384\n",
      "iteration 2100 / 5000: loss 2.284947\n",
      "iteration 2200 / 5000: loss 2.268976\n",
      "iteration 2300 / 5000: loss 2.274463\n",
      "iteration 2400 / 5000: loss 2.265744\n",
      "iteration 2500 / 5000: loss 2.278983\n",
      "iteration 2600 / 5000: loss 2.265891\n",
      "iteration 2700 / 5000: loss 2.272047\n",
      "iteration 2800 / 5000: loss 2.271091\n",
      "iteration 2900 / 5000: loss 2.256798\n",
      "iteration 3000 / 5000: loss 2.271892\n",
      "iteration 3100 / 5000: loss 2.261373\n",
      "iteration 3200 / 5000: loss 2.273079\n",
      "iteration 3300 / 5000: loss 2.270296\n",
      "iteration 3400 / 5000: loss 2.270972\n",
      "iteration 3500 / 5000: loss 2.262315\n",
      "iteration 3600 / 5000: loss 2.265795\n",
      "iteration 3700 / 5000: loss 2.270907\n",
      "iteration 3800 / 5000: loss 2.254316\n",
      "iteration 3900 / 5000: loss 2.263686\n",
      "iteration 4000 / 5000: loss 2.275910\n",
      "iteration 4100 / 5000: loss 2.267567\n",
      "iteration 4200 / 5000: loss 2.261070\n",
      "iteration 4300 / 5000: loss 2.262615\n",
      "iteration 4400 / 5000: loss 2.252818\n",
      "iteration 4500 / 5000: loss 2.266882\n",
      "iteration 4600 / 5000: loss 2.281692\n",
      "iteration 4700 / 5000: loss 2.272527\n",
      "iteration 4800 / 5000: loss 2.263273\n",
      "iteration 4900 / 5000: loss 2.279342\n",
      "iteration 0 / 5000: loss 775.078587\n",
      "iteration 100 / 5000: loss 6.851008\n",
      "iteration 200 / 5000: loss 2.124945\n",
      "iteration 300 / 5000: loss 2.091128\n",
      "iteration 400 / 5000: loss 2.111193\n",
      "iteration 500 / 5000: loss 2.071182\n",
      "iteration 600 / 5000: loss 2.064318\n",
      "iteration 700 / 5000: loss 2.102077\n",
      "iteration 800 / 5000: loss 2.139733\n",
      "iteration 900 / 5000: loss 2.107871\n",
      "iteration 1000 / 5000: loss 2.121478\n",
      "iteration 1100 / 5000: loss 2.088639\n",
      "iteration 1200 / 5000: loss 2.141284\n",
      "iteration 1300 / 5000: loss 2.121886\n",
      "iteration 1400 / 5000: loss 2.105929\n",
      "iteration 1500 / 5000: loss 2.139544\n",
      "iteration 1600 / 5000: loss 2.109752\n",
      "iteration 1700 / 5000: loss 2.112550\n",
      "iteration 1800 / 5000: loss 2.156427\n",
      "iteration 1900 / 5000: loss 2.095313\n",
      "iteration 2000 / 5000: loss 2.066555\n",
      "iteration 2100 / 5000: loss 2.090316\n",
      "iteration 2200 / 5000: loss 2.118185\n",
      "iteration 2300 / 5000: loss 2.113618\n",
      "iteration 2400 / 5000: loss 2.106248\n",
      "iteration 2500 / 5000: loss 2.109369\n",
      "iteration 2600 / 5000: loss 2.116606\n",
      "iteration 2700 / 5000: loss 2.079995\n",
      "iteration 2800 / 5000: loss 2.100355\n",
      "iteration 2900 / 5000: loss 2.047614\n",
      "iteration 3000 / 5000: loss 2.041325\n",
      "iteration 3100 / 5000: loss 2.132642\n",
      "iteration 3200 / 5000: loss 2.111674\n",
      "iteration 3300 / 5000: loss 2.088504\n",
      "iteration 3400 / 5000: loss 2.132580\n",
      "iteration 3500 / 5000: loss 2.064613\n",
      "iteration 3600 / 5000: loss 2.101178\n",
      "iteration 3700 / 5000: loss 2.069129\n",
      "iteration 3800 / 5000: loss 2.136634\n",
      "iteration 3900 / 5000: loss 2.114376\n",
      "iteration 4000 / 5000: loss 2.113660\n",
      "iteration 4100 / 5000: loss 2.067865\n",
      "iteration 4200 / 5000: loss 2.108971\n",
      "iteration 4300 / 5000: loss 2.084427\n",
      "iteration 4400 / 5000: loss 2.120351\n",
      "iteration 4500 / 5000: loss 2.085695\n",
      "iteration 4600 / 5000: loss 2.044767\n",
      "iteration 4700 / 5000: loss 2.115762\n",
      "iteration 4800 / 5000: loss 2.139480\n",
      "iteration 4900 / 5000: loss 2.108921\n",
      "iteration 0 / 5000: loss 15447.620521\n",
      "iteration 100 / 5000: loss 2.265815\n",
      "iteration 200 / 5000: loss 2.286145\n",
      "iteration 300 / 5000: loss 2.271469\n",
      "iteration 400 / 5000: loss 2.290073\n",
      "iteration 500 / 5000: loss 2.278201\n",
      "iteration 600 / 5000: loss 2.260544\n",
      "iteration 700 / 5000: loss 2.269511\n",
      "iteration 800 / 5000: loss 2.283218\n",
      "iteration 900 / 5000: loss 2.282075\n",
      "iteration 1000 / 5000: loss 2.291754\n",
      "iteration 1100 / 5000: loss 2.281710\n",
      "iteration 1200 / 5000: loss 2.271263\n",
      "iteration 1300 / 5000: loss 2.278602\n",
      "iteration 1400 / 5000: loss 2.286225\n",
      "iteration 1500 / 5000: loss 2.282077\n",
      "iteration 1600 / 5000: loss 2.280706\n",
      "iteration 1700 / 5000: loss 2.280797\n",
      "iteration 1800 / 5000: loss 2.289905\n",
      "iteration 1900 / 5000: loss 2.273484\n",
      "iteration 2000 / 5000: loss 2.279153\n",
      "iteration 2100 / 5000: loss 2.271454\n",
      "iteration 2200 / 5000: loss 2.270216\n",
      "iteration 2300 / 5000: loss 2.265980\n",
      "iteration 2400 / 5000: loss 2.275085\n",
      "iteration 2500 / 5000: loss 2.276946\n",
      "iteration 2600 / 5000: loss 2.282945\n",
      "iteration 2700 / 5000: loss 2.266543\n",
      "iteration 2800 / 5000: loss 2.282703\n",
      "iteration 2900 / 5000: loss 2.280024\n",
      "iteration 3000 / 5000: loss 2.292553\n",
      "iteration 3100 / 5000: loss 2.280175\n",
      "iteration 3200 / 5000: loss 2.255885\n",
      "iteration 3300 / 5000: loss 2.275571\n",
      "iteration 3400 / 5000: loss 2.284170\n",
      "iteration 3500 / 5000: loss 2.270695\n",
      "iteration 3600 / 5000: loss 2.291874\n",
      "iteration 3700 / 5000: loss 2.269755\n",
      "iteration 3800 / 5000: loss 2.288839\n",
      "iteration 3900 / 5000: loss 2.277869\n",
      "iteration 4000 / 5000: loss 2.278996\n",
      "iteration 4100 / 5000: loss 2.283885\n",
      "iteration 4200 / 5000: loss 2.267544\n",
      "iteration 4300 / 5000: loss 2.279555\n",
      "iteration 4400 / 5000: loss 2.275416\n",
      "iteration 4500 / 5000: loss 2.277384\n",
      "iteration 4600 / 5000: loss 2.290361\n",
      "iteration 4700 / 5000: loss 2.281381\n",
      "iteration 4800 / 5000: loss 2.268010\n",
      "iteration 4900 / 5000: loss 2.266170\n",
      "lr 1.000000e-07 reg 5.000000e+04 train accuracy: 0.327020 val accuracy: 0.338000\n",
      "lr 1.000000e-07 reg 1.000000e+06 train accuracy: 0.251000 val accuracy: 0.266000\n",
      "lr 5.000000e-07 reg 5.000000e+04 train accuracy: 0.321224 val accuracy: 0.333000\n",
      "lr 5.000000e-07 reg 1.000000e+06 train accuracy: 0.203857 val accuracy: 0.214000\n",
      "best validation accuracy achieved during cross-validation: 0.338000\n"
     ]
    }
   ],
   "source": [
    "# Use the validation set to tune hyperparameters (regularization strength and\n",
    "# learning rate). You should experiment with different ranges for the learning\n",
    "# rates and regularization strengths; if you are careful you should be able to\n",
    "# get a classification accuracy of over 0.35 on the validation set.\n",
    "from cs231n.classifiers import Softmax\n",
    "results = {}\n",
    "best_val = -1\n",
    "best_softmax = None\n",
    "learning_rates = [1e-7, 5e-7]\n",
    "regularization_strengths = [5e4, 1e6]\n",
    "\n",
    "################################################################################\n",
    "# TODO:                                                                        #\n",
    "# Use the validation set to set the learning rate and regularization strength. #\n",
    "# This should be identical to the validation that you did for the SVM; save    #\n",
    "# the best trained softmax classifer in best_softmax.                          #\n",
    "################################################################################\n",
    "for rate in learning_rates:\n",
    "    for reg in regularization_strengths:\n",
    "        sm = Softmax()\n",
    "        loss_hist = sm.train(X_train, y_train, learning_rate=rate, reg=reg, num_iters=5000, verbose=True)\n",
    "        y_train_pred = sm.predict(X_train)\n",
    "        acc_tr = np.mean(y_train == y_train_pred)\n",
    "        y_val_pred = sm.predict(X_val)\n",
    "        acc_val = np.mean(y_val == y_val_pred)\n",
    "        \n",
    "        results[(rate, reg)] = (acc_tr, acc_val)\n",
    "        if acc_val > best_val:\n",
    "            best_val = acc_val\n",
    "            best_softmax = sm\n",
    "################################################################################\n",
    "#                              END OF YOUR CODE                                #\n",
    "################################################################################\n",
    "    \n",
    "# Print out results.\n",
    "for lr, reg in sorted(results):\n",
    "    train_accuracy, val_accuracy = results[(lr, reg)]\n",
    "    print ('lr %e reg %e train accuracy: %f val accuracy: %f' % (\n",
    "                lr, reg, train_accuracy, val_accuracy))\n",
    "    \n",
    "print ('best validation accuracy achieved during cross-validation: %f' % best_val)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuAAAAIWCAYAAAAWFTgNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfbRlZX0n+O9PSnyLCkhpaAobEssk6Ep8qSC2kyyjBtF2hPRgByeJNYZezGQw2unuSSCdFWZ8WUs76Zi4OtpDRxQyRqRRA2MbkUGNK2kVClReREMFjJQQKS0kduyghb/54+yrh/IW3CrqPvfWvZ/PWmeds5/97H2e7cZT39r17N+u7g4AADDGQ1Z6AAAAsJ4I4AAAMJAADgAAAwngAAAwkAAOAAADCeAAADDQhpUewGhHHnlkH3vssSs9DAAA1rBrrrnmq929cbF16y6AH3vssdm2bdtKDwMAgDWsqv5mb+tMQQEAgIGWLYBX1flVdWdV3bBH+69W1Req6saq+ndz7edU1fZp3Qvn2k+e2rZX1dlz7cdV1aeq6uaqek9VHbpcxwIAAAfKcl4Bf2eSk+cbqupnkpyS5Me7+ylJfndqPz7J6UmeMm3z1qo6pKoOSfKHSV6U5PgkL5/6Jsmbkry5uzcnuSvJGct4LAAAcEAsWwDv7o8n2bVH868keWN33zP1uXNqPyXJRd19T3ffmmR7khOm1/buvqW7v5XkoiSnVFUleV6SS6btL0hy6nIdCwAAHCij54A/OclPTVNH/ryqfnJqPzrJbXP9dkxte2t/XJKvd/fuPdoXVVVnVtW2qtq2c+fOA3QoAACw70YH8A1JDk9yYpL/I8nF09XsWqRv70f7orr7vO7e0t1bNm5ctBoMAAAMMboM4Y4k7+vuTnJVVX0nyZFT+zFz/TYluX36vFj7V5McVlUbpqvg8/0BAGDVGn0F/E8zm7udqnpykkMzC9OXJTm9qh5WVccl2ZzkqiRXJ9k8VTw5NLMbNS+bAvxHk5w27XdrkkuHHgkAAOyHZbsCXlXvTvLcJEdW1Y4k5yY5P8n5U2nCbyXZOoXpG6vq4iSfS7I7yVndfe+0n1cluTzJIUnO7+4bp6/4jSQXVdXrk3w6yduX61gAAOBAqVn+XT+2bNnSnoQJAMByqqprunvLYus8CRMAAAYSwAEAYCABHAAABhLAAQBgIAEcAAAGEsABAGAgARwAAAYSwAEAYCABHAAABhLAB/itP70+z/vdj630MAAAWAUE8AG+tfs7+e/fvnelhwEAwCoggAMAwEACOAAADCSAAwDAQAI4AAAMJIAP0r3SIwAAYDUQwAeo1EoPAQCAVUIABwCAgQRwAAAYSAAHAICBBHAAABhIAAcAgIEE8EE66hACACCAD1GqEAIAMBHAAQBgIAEcAAAGEsABAGAgARwAAAYSwAEAYCABfJBWhRAAgAjgQyhDCADAAgEcAAAGEsABAGAgARwAAAYSwAEAYCABfBBFUAAASATwQZRBAQBgRgAHAICBBHAAABhIAAcAgIEEcAAAGEgABwCAgQTwQVodQgAAIoAPUaoQAgAwEcABAGAgARwAAAYSwAEAYCABHAAABlq2AF5V51fVnVV1wyLr/k1VdVUdOS1XVb2lqrZX1XVV9Yy5vlur6ubptXWu/ZlVdf20zVuq3OoIAMDqt5xXwN+Z5OQ9G6vqmCQ/m+RLc80vSrJ5ep2Z5G1T3yOSnJvkWUlOSHJuVR0+bfO2qe/Cdt/3XauLOoQAACxjAO/ujyfZtciqNyf59dw3kZ6S5MKe+WSSw6rqqCQvTHJFd+/q7ruSXJHk5GndY7r7E93dSS5McupyHcuD5dI8AAALhs4Br6qXJvlyd392j1VHJ7ltbnnH1HZ/7TsWad/b955ZVduqatvOnTsfxBEAAMCDMyyAV9Ujk/zbJL+92OpF2no/2hfV3ed195bu3rJx48alDBcAAJbFyCvgP5zkuCSfraovJtmU5Nqq+sHMrmAfM9d3U5LbH6B90yLtAACwqg0L4N19fXc/vruP7e5jMwvRz+juv01yWZJXTNVQTkxyd3ffkeTyJCdV1eHTzZcnJbl8WveNqjpxqn7yiiSXjjoWAADYX8tZhvDdST6R5EeqakdVnXE/3T+Y5JYk25P8pyT/e5J0964kr0ty9fR67dSWJL+S5I+mbf46yZ8tx3EAAMCBtGG5dtzdL3+A9cfOfe4kZ+2l3/lJzl+kfVuSpz64UY7TqhACABBPwhzCI4IAAFgggAMAwEACOAAADCSAAwDAQAI4AAAMJIAPoggKAACJAD5ERRkUAABmBHAAABhIAAcAgIEEcAAAGEgABwCAgQRwAAAYSAAfpFshQgAABPAhShVCAAAmAjgAAAwkgAMAwEACOAAADCSAAwDAQAI4AAAMJIAPogghAACJAD6EKoQAACwQwAEAYCABHAAABhLAAQBgIAEcAAAGEsABAGAgAXyQVocQAIAI4ENUKUQIAMCMAA4AAAMJ4AAAMJAADgAAAwngAAAwkAA+SCuDAgBABHAAABhKAAcAgIEEcAAAGEgABwCAgQRwAAAYSAAHAICBBPBBFCEEACARwIeoWukRAACwWgjgAAAwkAAOAAADCeAAADCQAA4AAAMJ4AAAMJAAPoo6hAAARAAfoqIOIQAAMwI4AAAMtGwBvKrOr6o7q+qGubbfqarPV9V1VfX+qjpsbt05VbW9qr5QVS+caz95atteVWfPtR9XVZ+qqpur6j1VdehyHQsAABwoy3kF/J1JTt6j7YokT+3uH0/yV0nOSZKqOj7J6UmeMm3z1qo6pKoOSfKHSV6U5PgkL5/6Jsmbkry5uzcnuSvJGct4LAAAcEAsWwDv7o8n2bVH24e7e/e0+Mkkm6bPpyS5qLvv6e5bk2xPcsL02t7dt3T3t5JclOSUqqokz0tyybT9BUlOXa5jAQCAA2Ul54D/cpI/mz4fneS2uXU7pra9tT8uydfnwvxC+6Kq6syq2lZV23bu3HmAhg8AAPtuRQJ4Vf3bJLuTvGuhaZFuvR/ti+ru87p7S3dv2bhx474O94BQhRAAgCTZMPoLq2prkpckeX53L+TSHUmOmeu2Kcnt0+fF2r+a5LCq2jBdBZ/vv+qUKoQAAEyGXgGvqpOT/EaSl3b3N+dWXZbk9Kp6WFUdl2RzkquSXJ1k81Tx5NDMbtS8bAruH01y2rT91iSXjjoOAADYX8tZhvDdST6R5EeqakdVnZHkPyR5dJIrquozVfUfk6S7b0xycZLPJflQkrO6+97p6varklye5KYkF099k1mQ/1dVtT2zOeFvX65jAQCAA2XZpqB098sXad5rSO7uNyR5wyLtH0zywUXab8msSgoAABw0PAkTAAAGEsAH+d79pgAArGcC+ACKoAAAsEAABwCAgQRwAAAYSAAHAICBBHAAABhIAAcAgIEE8EEUIQQAIBHAhyh1CAEAmAjgAAAwkAAOAAADCeAAADCQAA4AAAMJ4AAAMJAAPkirQwgAQATwIUodQgAAJgI4AAAMJIADAMBAAjgAAAwkgAMAwEACOAAADCSAD9JRhxAAAAF8CEUIAQBYIIADAMBAAjgAAAwkgAMAwEACOAAADCSAD9KKoAAAEAF8DGVQAACYCOAAADCQAA4AAAMJ4AAAMJAADgAAAwngAAAwkAA+iCqEAAAkAvgQpQ4hAAATARwAAAYSwAEAYCABHAAABhLAAQBgIAEcAAAGEsBHUYcQAIAI4EOUKoQAAEwEcAAAGEgABwCAgQRwAAAYSAAHAICBli2AV9X5VXVnVd0w13ZEVV1RVTdP74dP7VVVb6mq7VV1XVU9Y26brVP/m6tq61z7M6vq+mmbt1S51REAgNVvOa+AvzPJyXu0nZ3kyu7enOTKaTlJXpRk8/Q6M8nbkllgT3JukmclOSHJuQuhfepz5tx2e37XqtLqEAIAkGUM4N398SS79mg+JckF0+cLkpw6135hz3wyyWFVdVSSFya5ort3dfddSa5IcvK07jHd/Ynu7iQXzu1r1XFpHgCABaPngD+hu+9Ikun98VP70Ulum+u3Y2q7v/Ydi7QDAMCqtlpuwlzsInHvR/viO686s6q2VdW2nTt37ucQAQDgwRsdwL8yTR/J9H7n1L4jyTFz/TYluf0B2jct0r6o7j6vu7d095aNGzc+6IMAAID9NTqAX5ZkoZLJ1iSXzrW/YqqGcmKSu6cpKpcnOamqDp9uvjwpyeXTum9U1YlT9ZNXzO0LAABWrQ3LteOqeneS5yY5sqp2ZFbN5I1JLq6qM5J8KcnLpu4fTPLiJNuTfDPJK5Oku3dV1euSXD31e213L9zY+SuZVVp5RJI/m16rViuCAgBAljGAd/fL97Lq+Yv07SRn7WU/5yc5f5H2bUme+mDGOIoK5QAALFgtN2ECAMC6IIADAMBAAjgAAAwkgAMAwEACOAAADCSAD6IKIQAAiQA+REUdQgAAZgRwAAAYSAAHAICBBHAAABhIAAcAgIEEcAAAGEgAH6RbIUIAAATwIUoVQgAAJgI4AAAMJIADAMBAAjgAAAwkgAMAwEAC+CBqoAAAkAjgQyiCAgDAAgEcAAAGEsABAGAgARwAAAYSwAEAYCABHAAABhLAB2l1CAEAiAA+RilECADAjAAOAAADCeAAADCQAA4AAAMJ4AAAMJAADgAAAwngAAAwkAA+gCKEAAAsEMABAGAgARwAAAZ6wABeVU+uqiur6oZp+cer6reWf2gAALD2LOUK+H9Kck6SbydJd1+X5PTlHBQAAKxVSwngj+zuq/Zo270cgwEAgLVuKQH8q1X1w0k6SarqtCR3LOuo1qjuXukhAACwwjYsoc9ZSc5L8qNV9eUktyb5xWUd1RpT6hACADB5wADe3bckeUFVPSrJQ7r7G8s/LAAAWJseMIBX1W/vsZwk6e7XLtOYAABgzVrKFJS/n/v88CQvSXLT8gwHAADWtqVMQfn388tV9btJLlu2EQEAwBq2P0/CfGSSHzrQA1kPFEEBAGApc8Cvz1SCMMkhSTYmMf97H1SUQQEAYGYpc8BfMvd5d5KvdLcH8QAAwH7YawCvqiOmj3uWHXxMVaW7dy3fsAAAYG26vyvg12Q29WSx+ROdBzEPvKp+Lcm/mPZzfZJXJjkqyUVJjkhybZJf6u5vVdXDklyY5JlJvpbk57v7i9N+zklyRpJ7k7y6uy/f3zEBAMAIe70Js7uP6+4fmt73fD2Y8H10klcn2dLdT81sXvnpSd6U5M3dvTnJXZkF60zvd3X3k5K8eeqXqjp+2u4pSU5O8taqOmR/xwUAACMsqQpKVR1eVSdU1U8vvB7k925I8oiq2pBZVZU7kjwvySXT+guSnDp9PmVazrT++TV7GtApSS7q7nu6+9Yk25Oc8CDHBQAAy2opVVD+RZLXJNmU5DNJTkzyicwC8z7r7i9PtcS/lOS/J/lwZtNdvj53c+eOJEdPn49Octu07e6qujvJ46b2T87ten6bPY/hzCRnJskTn/jE/Rn2AaEKIQAAS7kC/pokP5nkb7r7Z5I8PcnO/f3Cqjo8s6vXxyX5R0keleRFi3RdyKt7m4O+t/bvb+w+r7u3dPeWjRs37vugH6RShRAAgMlSAvg/dPc/JElVPay7P5/kRx7Ed74gya3dvbO7v53kfUn+SZLDpikpyexq++3T5x1Jjpm+f0OSxybZNd++yDYAALAqLSWA76iqw5L8aZIrqurSPLig+6UkJ1bVI6e53M9P8rkkH01y2tRna5JLp8+XTcuZ1n+ku3tqP72qHlZVxyXZnOSqBzEuAABYdg84B7y7f276+H9W1UczuwL9of39wu7+VFVdklmpwd1JPp3kvCT/JclFVfX6qe3t0yZvT/LHVbU9syvfp0/7ubGqLs4svO9OclZ337u/4wIAgBGWchPmHyR5T3f/1+7+8wPxpd19bpJz92i+JYtUMZmmv7xsL/t5Q5I3HIgxAQDACEuZgnJtkt+qqu1V9TtVtWW5BwUAAGvVAwbw7r6gu1+c2dXpv0rypqq6edlHtgbNpq4DALCeLelBPJMnJfnRJMcm+fyyjGaNUoUQAIAFDxjAq2rhivdrk9yQ5Jnd/T8u+8gAAGANesCbMJPcmuTZ3f3V5R4MAACsdUspQ/gfRwwEAADWg32ZAw4AADxIAjgAAAy0lJswf7iqHjZ9fm5VvXp6ND37SBFCAACWcgX8vUnuraonZfZY+OOS/MmyjmqNKXUIAQCYLCWAf6e7dyf5uSS/392/luSo5R0WAACsTUsJ4N+uqpcn2ZrkA1PbQ5dvSAAAsHYtJYC/Msmzk7yhu2+tquOS/D/LOywAAFibllIH/HNJXp0kVXV4kkd39xuXe2AAALAWLaUKyseq6jFVdUSSzyZ5R1X93vIPbe1pZVAAANa9pUxBeWx3/12Sf5bkHd39zCQvWN5hrS2lDAoAAJOlBPANVXVUkn+e792ECQAA7IelBPDXJrk8yV9399VV9UNJbl7eYQEAwNq0lJsw/3OS/zy3fEuS/2k5BwUAAGvVUm7C3FRV76+qO6vqK1X13qraNGJwAACw1ixlCso7klyW5B8lOTrJ/zu1AQAA+2gpAXxjd7+ju3dPr3cm2bjM41qTOuoQAgCsd0sJ4F+tql+sqkOm1y8m+dpyDwwAANaipQTwX86sBOHfJrkjyWmZPZ4eAADYRw8YwLv7S9390u7e2N2P7+5TM3soDwAAsI+WcgV8Mf/qgI4CAADWif0N4J6tDgAA+2F/A7hyHgAAsB/2+iTMqvpGFg/aleQRyzaiNaz9tQUAYN3bawDv7kePHMhaVibsAAAw2d8pKAAAwH4QwAEAYCABHAAABhLAAQBgIAEcAAAGEsABAGAgAXyA8uBQAAAmAjgAAAwkgAMAwEACOAAADCSAAwDAQAL4QN0rPQIAAFaaAD5AKYICAMBEAAcAgIEEcAAAGEgABwCAgQRwAAAYSAAHAICBViSAV9VhVXVJVX2+qm6qqmdX1RFVdUVV3Ty9Hz71rap6S1Vtr6rrquoZc/vZOvW/uaq2rsSx7IuOOoQAAOvdSl0B/4MkH+ruH03yE0luSnJ2kiu7e3OSK6flJHlRks3T68wkb0uSqjoiyblJnpXkhCTnLoT21UYVQgAAFgwP4FX1mCQ/neTtSdLd3+ruryc5JckFU7cLkpw6fT4lyYU988kkh1XVUUlemOSK7t7V3XcluSLJyQMPBQAA9tlKXAH/oSQ7k7yjqj5dVX9UVY9K8oTuviNJpvfHT/2PTnLb3PY7pra9tQMAwKq1EgF8Q5JnJHlbdz89yd/ne9NNFrPYDI6+n/bv30HVmVW1raq27dy5c1/HCwAAB8xKBPAdSXZ096em5UsyC+RfmaaWZHq/c67/MXPbb0py+/20f5/uPq+7t3T3lo0bNx6wAwEAgH01PIB3998mua2qfmRqen6SzyW5LMlCJZOtSS6dPl+W5BVTNZQTk9w9TVG5PMlJVXX4dPPlSVMbAACsWhtW6Ht/Ncm7qurQJLckeWVmfxm4uKrOSPKlJC+b+n4wyYuTbE/yzalvuntXVb0uydVTv9d2965xh7DvWhVCAIB1b0UCeHd/JsmWRVY9f5G+neSsvezn/CTnH9jRHXilDiEAABNPwgQAgIEEcAAAGEgABwCAgQRwAAAYSAAHAICBBPCBVCEEAEAAH6CiDiEAADMCOAAADCSAAwDAQAI4AAAMJIADAMBAAvhA3eqgAACsdwL4AKUICgAAEwEcAAAGEsABAGAgARwAAAYSwAEAYCABHAAABhLAB1KEEAAAARwAAAYSwAEAYCABHAAABhLAAQBgIAEcAAAGEsABAGAgAXygVocQAGDdE8AHqKqVHgIAAKuEAA4AAAMJ4AAAMJAADgAAAwngAAAwkAA+kiooAADrngA+gBooAAAsEMABAGAgARwAAAYSwAEAYCABHAAABhLAAQBgIAF8oFaHEABg3RPAByh1CAEAmAjgAAAwkAAOAAADCeAAADCQAA4AAAMJ4AAAMJAAPlCrQggAsO4J4AOoQggAwAIBHAAABhLAAQBgoBUL4FV1SFV9uqo+MC0fV1Wfqqqbq+o9VXXo1P6waXn7tP7YuX2cM7V/oapeuDJHAgAAS7eSV8Bfk+SmueU3JXlzd29OcleSM6b2M5Lc1d1PSvLmqV+q6vgkpyd5SpKTk7y1qg4ZNHYAANgvKxLAq2pTkn+a5I+m5UryvCSXTF0uSHLq9PmUaTnT+udP/U9JclF339PdtybZnuSEMUcAAAD7Z6WugP9+kl9P8p1p+XFJvt7du6flHUmOnj4fneS2JJnW3z31/277ItvcR1WdWVXbqmrbzp07D+Rx7BNVCAEAGB7Aq+olSe7s7mvmmxfp2g+w7v62uW9j93ndvaW7t2zcuHGfxnsgzC7YAwBAsmEFvvM5SV5aVS9O8vAkj8nsivhhVbVhusq9KcntU/8dSY5JsqOqNiR5bJJdc+0L5rcBAIBVafgV8O4+p7s3dfexmd1E+ZHu/oUkH01y2tRta5JLp8+XTcuZ1n+ku3tqP32qknJcks1Jrhp0GAAAsF9W4gr43vxGkouq6vVJPp3k7VP725P8cVVtz+zK9+lJ0t03VtXFST6XZHeSs7r73vHDBgCApVvRAN7dH0vysenzLVmkikl3/0OSl+1l+zckecPyjRAAAA4sT8IcaDZzBgCA9UwAH0ARFAAAFgjgAAAwkAAOAAADCeAAADCQAA4AAAMJ4AAAMJAAPpAihAAACOADqEIIAMACARwAAAYSwAEAYCABHAAABhLAAQBgIAEcAAAGEsAHanUIAQDWPQF8hFKIEACAGQEcAAAGEsABAGAgARwAAAYSwAEAYCABHAAABhLAB+qoQwgAsN4J4AMoQggAwAIBHAAABhLAAQBgIAEcAAAGEsABAGAgAXwkRVAAANY9AXyAUgYFAICJAA4AAAMJ4AAAMJAADgAAAwngAAAwkAAOAAADCeADqUIIAIAAPkBFHUIAAGYEcAAAGEgABwCAgQRwAAAYSAAHAICBBHAAABhIAB+o1SEEAFj3BPABShVCAAAmAjgAAAwkgAMAwEACOAAADCSAAwDAQAI4AAAMJIAP1FGHEABgvRsewKvqmKr6aFXdVFU3VtVrpvYjquqKqrp5ej98aq+qektVba+q66rqGXP72jr1v7mqto4+lqVShRAAgAUrcQV8d5J/3d0/luTEJGdV1fFJzk5yZXdvTnLltJwkL0qyeXqdmeRtySywJzk3ybOSnJDk3IXQDgAAq9XwAN7dd3T3tdPnbyS5KcnRSU5JcsHU7YIkp06fT0lyYc98MslhVXVUkhcmuaK7d3X3XUmuSHLywEMBAIB9tqJzwKvq2CRPT/KpJE/o7juSWUhP8vip29FJbpvbbMfUtrf2xb7nzKraVlXbdu7ceSAPAQAA9smKBfCq+oEk703yL7v77+6v6yJtfT/t39/YfV53b+nuLRs3btz3wQIAwAGyIgG8qh6aWfh+V3e/b2r+yjS1JNP7nVP7jiTHzG2+Kcnt99O+arUiKAAA695KVEGpJG9PclN3/97cqsuSLFQy2Zrk0rn2V0zVUE5Mcvc0ReXyJCdV1eHTzZcnTW2rTimDAgDAZMMKfOdzkvxSkuur6jNT228meWOSi6vqjCRfSvKyad0Hk7w4yfYk30zyyiTp7l1V9bokV0/9Xtvdu8YcAgAA7J/hAby7/yJ7L439/EX6d5Kz9rKv85Ocf+BGBwAAy8uTMAEAYCABHAAABhLAAQBgIAF8IFUIAQAQwAeovd5zCgDAeiOAAwDAQAI4AAAMJIADAMBAAjgAAAwkgAMAwEAC+EDdChECAKx3AvgIqhACADARwAEAYCABHAAABhLAAQBgIAEcAAAGEsABAGAgAXwgVQgBABDAB1CFEACABQI4AAAMJIADAMBAAjgAAAwkgAMAwEACOAAADCSAD/CQmtVBUYYQAAABfICHTP8r3yuBAwCsewL4AAtXwL8jgAMArHsC+ADfDeDfEcABANY7AXyAQx6ycAV8hQcCAMCKE8AHmPK3KSgAAAjgIyxMQbnXJXAAgHVPAB9AGUIAABYI4AMslCE0BQUAAAF8gO9OQRHAAQDWPQF8gO9NQRHAAQDWOwF8gIUyhPd+Z4UHAgDAihPAByhlCAEAmAjgA3gUPQAACwTwAb77JExTUAAA1j0BfABPwgQAYIEAPoAyhAAALBDAB1CGEACABQL4AOaAAwCwQAAfYKEMoSkoAAAI4AOYggIAwAIBfABPwgQAYIEAPoAyhAAALBDAB/AkTAAAFhz0AbyqTq6qL1TV9qo6e6XHsxgBHACABQd1AK+qQ5L8YZIXJTk+ycur6viVHdX3W5gDvvteARwAYL3bsNIDeJBOSLK9u29Jkqq6KMkpST63oqPaw+N+4NAkySdu+Vp+7KjHJPleaUJm/OMAHFz8hgEHi8MeeWiOPuwRKz2M+zjYA/jRSW6bW96R5FkrNJa9euShG/LkJ/xA3nftl/O+a7+80sMBAFg3/vmWTfl3p/3ESg/jPg72AL7YNZjvu5ZaVWcmOTNJnvjEJy73mBZ14S8/K9d/+e509/cPkCSLn0xg9fEbBhxMVtvV7+TgD+A7khwzt7wpye17duru85KclyRbtmxZkT87fvCxD88PPvbhK/HVAACsIgf1TZhJrk6yuaqOq6pDk5ye5LIVHhMAAOzVQX0FvLt3V9Wrklye5JAk53f3jSs8LAAA2KuDOoAnSXd/MMkHV3ocAACwFAf7FBQAADioCOAAADCQAA4AAAMJ4AAAMJAADgAAAwngAAAwkAAOAAADCeAAADCQAA4AAAMJ4AAAMJAADgAAAwngAAAwkAAOAAADCeAAADCQAA4AAAMJ4AAAMJAADgAAA1V3r/QYhqqqnUn+ZgW++sgkX12B72Us53l9cJ7XPud4fXCe14eVOs//uLs3LrZi3QXwlVJV27p7y0qPg+XlPK8PzvPa5xyvD87z+rAaz7MpKAAAMJAADgAAAwng45y30gNgCOd5fXCe1z7neH1wnteHVXeezQEHAICBXAEHAICBBPABqurkqvpCVW2vqrNXejwsXVWdX1V3VtUNc21HVNUVVXXz9H741PyfmV8AAAf6SURBVF5V9ZbpPF9XVc+Y22br1P/mqtq6EsfC3lXVMVX10aq6qapurKrXTO3O9RpSVQ+vqquq6rPTef6/pvbjqupT0zl7T1UdOrU/bFrePq0/dm5f50ztX6iqF67MEbE3VXVIVX26qj4wLTvHa0xVfbGqrq+qz1TVtqntoPnNFsCXWVUdkuQPk7woyfFJXl5Vx6/sqNgH70xy8h5tZye5srs3J7lyWk5m53jz9DozyduS2Q9CknOTPCvJCUnOXfhRYNXYneRfd/ePJTkxyVnT/0+d67XlniTP6+6fSPK0JCdX1YlJ3pTkzdN5vivJGVP/M5Lc1d1PSvLmqV+m/zZOT/KUzH4f3jr91rN6vCbJTXPLzvHa9DPd/bS5EoMHzW+2AL78Tkiyvbtv6e5vJbkoySkrPCaWqLs/nmTXHs2nJLlg+nxBklPn2i/smU8mOayqjkrywiRXdPeu7r4ryRX5/lDPCuruO7r72unzNzL7g/voONdrynS+/tu0+NDp1Umel+SSqX3P87xw/i9J8vyqqqn9ou6+p7tvTbI9s996VoGq2pTknyb5o2m54hyvFwfNb7YAvvyOTnLb3PKOqY2D1xO6+45kFtySPH5q39u59t/AQWT6J+inJ/lUnOs1Z5qa8Jkkd2b2h+1fJ/l6d++eusyfs++ez2n93UkeF+d5tfv9JL+e5DvT8uPiHK9FneTDVXVNVZ05tR00v9kbRnzJOleLtCk9szbt7Vz7b+AgUVU/kOS9Sf5ld//d7ELY4l0XaXOuDwLdfW+Sp1XVYUnen+THFus2vTvPB5mqekmSO7v7mqp67kLzIl2d44Pfc7r79qp6fJIrqurz99N31Z1nV8CX344kx8wtb0py+wqNhQPjK9M/XWV6v3Nq39u59t/AQaCqHppZ+H5Xd79vanau16ju/nqSj2U25/+wqlq4IDV/zr57Pqf1j81sSprzvHo9J8lLq+qLmU35fF5mV8Sd4zWmu2+f3u/M7C/TJ+Qg+s0WwJff1Uk2T3dgH5rZTR2XrfCYeHAuS7Jwp/TWJJfOtb9iutv6xCR3T/8EdnmSk6rq8OnmjpOmNlaJac7n25Pc1N2/N7fKuV5DqmrjdOU7VfWIJC/IbL7/R5OcNnXb8zwvnP/TknykZw/PuCzJ6VMFjeMyu7HrqjFHwf3p7nO6e1N3H5vZn7cf6e5fiHO8plTVo6rq0QufM/utvSEH0W+2KSjLrLt3V9WrMjuhhyQ5v7tvXOFhsURV9e4kz01yZFXtyOxu6TcmubiqzkjypSQvm7p/MMmLM7tZ55tJXpkk3b2rql6X2V/GkuS13b3njZ2srOck+aUk10/zg5PkN+NcrzVHJblgqmbxkCQXd/cHqupzSS6qqtcn+XRmfxnL9P7HVbU9s6uipydJd99YVRcn+VxmFXTOmqa2sHr9RpzjteQJSd4/TRPckORPuvtDVXV1DpLfbE/CBACAgUxBAQCAgQRwAAAYSAAHAICBBHAAABhIAAcAgIEEcIAVVFX/bXo/tqr+5wO879/cY/m/Hsj9H2hV9b9U1X9Y6XEALDcBHGB1ODbJPgXwqZ71/blPAO/uf7KPYzqoLOF/D4BVQQAHWB3emOSnquozVfVrVXVIVf1OVV1dVddV1f+aJFX13Kr6aFX9SZLrp7Y/raprqurGqjpzantjkkdM+3vX1LZwtb2mfd9QVddX1c/P7ftjVXVJVX2+qt41PSX0PqY+b6qqq6rqr6rqp6b2+1zBrqoPVNVzF7572uaaqvr/quqEaT+3VNVL53Z/TFV9qKq+UFXnzu3rF6fv+0xV/d8LYXva72ur6lNJnn2gTgbAcvIkTIDV4ewk/6a7X5IkU5C+u7t/sqoeluQvq+rDU98Tkjy1u2+dln95eqLbI5JcXVXv7e6zq+pV3f20Rb7rnyV5WpKfSHLktM3Hp3VPT/KUJLcn+cvMnhL6F4vsY0N3n1BVL87sCbEveIDje1SSj3X3b1TV+5O8PsnPJjk+yQWZPSr6u8eW2dPqrq6q/5Lk75P8fJLndPe3q+qtSX4hyYXTfm/o7t9+gO8HWDUEcIDV6aQkP15Vp03Lj02yOcm3klw1F76T5NVV9XPT52Omfl+7n33/D0nePT1a+ytV9edJfjLJ30373pEkVfWZzKbGLBbA3ze9XzP1eSDfSvKh6fP1Se6ZwvT1e2x/RXd/bfr+901j3Z3kmZkF8iR5RJI7p/73JnnvEr4fYNUQwAFWp0ryq919+X0aZ1M6/n6P5RckeXZ3f7OqPpbk4UvY997cM/f53uz9z4l7FumzO/ed2jg/jm93d0+fv7OwfXd/p6rmv6NzXz2N94LuPmeRcfzD9BcJgIOGOeAAq8M3kjx6bvnyJL9SVQ9Nkqp6clU9apHtHpvkril8/2iSE+fWfXth+z18PMnPT/PMNyb56SRXHYBj+GKSp1XVQ6rqmMymk+yrn62qI6bpNKdmNg3myiSnVdXjk2Ra/48PwHgBVoQr4ACrw3VJdlfVZ5O8M8kfZDY149rpRsidmQXSPX0oyf9WVdcl+UKST86tOy/JdVV1bXf/wlz7+zO7YfGzmV1h/vXu/tspwD8Yf5nk1symmNyQ5Nr92MdfJPnjJE9K8ifdvS1Jquq3kny4qh6S5NtJzkryNw9yvAAror73L4IAAMByMwUFAAAGEsABAGAgARwAAAYSwAEAYCABHAAABhLAAQBgIAEcAAAGEsABAGCg/x8tEVtezVQZsQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x648 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# A useful debugging strategy is to plot the loss as a function of\n",
    "# iteration number:\n",
    "plt.plot(loss_hist)\n",
    "plt.xlabel('Iteration number')\n",
    "plt.ylabel('Loss value')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "softmax on raw pixels final test set accuracy: 0.346000\n"
     ]
    }
   ],
   "source": [
    "# evaluate on test set\n",
    "# Evaluate the best softmax on test set\n",
    "y_test_pred = best_softmax.predict(X_test)\n",
    "test_accuracy = np.mean(y_test == y_test_pred)\n",
    "print ('softmax on raw pixels final test set accuracy: %f' % (test_accuracy, ))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqwAAAGcCAYAAAAcUPIAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9e9Qs2Vne9+6q6u7vnDOjGa4OEpIcg00MmAAOFyfm7kCAYBNhYwwGgwO5GEwUr3CNHIQDkU0Q2IQ4NhhsB5C5yARQzMoihCQLHGwwEEjAS+GmC0IGBALNnHO+7q6qnT/609Szf/t0nRnNdJ+aT89vrVnTdaq6ate+1f7qffp5U845jDHGGGOMWSrNgy6AMcYYY4wxc3jBaowxxhhjFo0XrMYYY4wxZtF4wWqMMcYYYxaNF6zGGGOMMWbReMFqjDHGGGMWzbVesKaUPjKl9GsPuhzGmGc/KaXXpJT+xD3+/cNSSq9+iuf6Bymlr3rmSmeMeTbjOeH+XOsFqzHGnJqc84/mnN/rQZfDXB+O/XFkzNszXrAa8wySUuoedBnMcnB/MMY8KK7b/HMtFqxXf41+WUrpF1JKb04p/f2U0sU9jvvSlNIvp5Qeuzr2P5B9n51S+rGU0tdeneNXU0ofL/sfSSl9S0rpjSmlN6SUviql1J7rHs15SCk9P6X0vSml30op/XZK6RtTSu+RUvqRq+03pZS+I6X0qHznNSmlL0kp/VxE3L5uk4Qp+CDOM5Qe3as/pJQ+IKX001dzz3dFRDU/mevJU51TUkrfFhEviIhXpZQeTyl98YO9A3MK5uaElNK/n1L6v1NKv5tS+r9SSu8n+56bUvrHV/3pV1NKXyj7XppSemVK6dtTSm+JiM8+602dmGuxYL3iMyLi4yLiPSLiD0XES+5xzC9HxIdFxCMR8ZUR8e0ppXeT/R8SEa+OiHeOiK+JiG9JKaWrff8wIvqIeM+I+ICI+NiI+Nxn/jbMg+LqD5D/OSJeGxG/PyKeFxHfGREpIl4WEc+NiD8cEc+PiJfi638uIj4xIh7NOffnKbF5ADyZeSZC+kMc5tnvi4hvi4h3jIjviYhPOXlJzQPnbZlTcs6fGRGvi4hPyjk/lHP+mrMX3JyUlNI6jswJKaUPjIhvjYj/OCLeKSL+bkT8QEppk1JqIuJVEfGzcehLHxMRL04pfZyc/k9FxCvjMPd8x1lu6ExcpwXrN+acX59z/p2I+Oo4PDAKcs7fk3P+9ZzzmHP+roj4xYj4YDnktTnnb845D3FYoL5bRPy+lNLvi4iPj4gX55xv55x/MyK+PiI+7dQ3Zc7KB8fhAfJFV+18mXP+sZzzL+Wc/9ec8zbn/FsR8XUR8RH47jdc9b+7Zy+1OSf3nWeu0P7woRGxioi/mXPe55xfGRE/eabymgfL05lTzPVlbk74vIj4uznnf55zHnLO/zAitlff+aCIeJec81/LOe9yzr8SEd8c5Vrkx3PO33e1zrlWz6PrFLp8vXx+bRwmiYKU0mdFxF+Jw1+6EREPxeFt6lv5V2/9kHO+c/Vy9aE4/AW0iog3Ti9co8E1zbOf58fhj5biDWlK6V0j4hvi8Hb+4Ti0/ZvxXfeFtw/uO8/c47jnRsQbcs4Z3zXXn6czp5jry9yc8MKI+Asppb8s+9ZX3xki4rkppd+VfW1E/KhsX9tn0XV6w/p8+fyCiPh13ZlSemEc/hL5goh4p5zzoxHx/8YhNHM/Xh+Hv3DeOef86NV/z8k5v88zU3SzEF4fES+4hwb1ZRGRI+L9cs7PiYg/H3W/yWHeHpidZwTtD2+MiOeJvOit3zXXn7d1TvF8cr2ZmxNeHxFfLWuNR3PON3PO/+hq369i38M550+Q81zbvnOdFqyfn1J695TSO0bEl0fEd2H/rTg05G9FRKSUPici3vfJnDjn/MaI+KGIeHlK6TkppeZKNO8QzvXiJ+Iwkfz1lNKtqx/U/DtxeAPyeET8bkrpeRHxRQ+ykOaBcr955l78eBz071949QOsF0UpRTLXl7d1TvmNiPgD5y2qOSNzc8I3R8R/klL6kHTgVkrpE1NKD8ehP73l6kedN1JKbUrpfVNKH/SA7uOsXKcF6yvisKj8lav/CgPenPMvRMTL49BRfiMi/khE/NOncP7PisNr+V+IQ+jmlXHQuJprwpV2+ZPi8MO610XEr0XEn43DD/Q+MCJ+LyL+SUR874Mqo3ngzM4z9yLnvIuIF8XhF7tvjkOfch96O+BpzCkvi4iXXP1K/L84X4nNOZibE3LO/yIOOtZvvNr3S1fHaX96/4j41Yh4U0T8vTj8kPzak0oJxbOTlNJrIuJzc84//KDLYowxxhhjnlmu0xtWY4wxxhhzDfGC1RhjjDHGLJprIQkwxhhjjDHXF79hNcYYY4wxi2Y2ccBf/Gs/Xrx+zXl84vMonyMimmZa+6amXAc3qdwe5K3uOJbnyTjvKMfWL4Of/NvhLNe531vlsvzHbVqrPTivnibh6LkytE/hz4iM+tNSNWiHv//SD38ynrNvE9/0FX+1uKFWrt11ZTcbhuGJzyOqoZd9h/PMFBm7WrW0w73npOVZFftYhXnUPlfuHMYy62oj10xNi33TdstGxX1rP+f4ybzRdNyuUcs+DhhLM2NtQHn+06/4L0/SV77giz+8uNJqvZ7KwzFSlPc+Y13qhGOrum/92sy4zKiBri37sVooNmhfjstBttnHWYayPBzf9y5rxD36TT7ejzu5ZHV9DkrdhX1/++U/drI55b/9tD95dE5pcdVB+zqK3yTe33Rswj72lZgZw9WYFjLqqe913iivyeej3kCDOSXpfDjTx3iVludBlTRy3hEVqFU7YE7JM31u2++LfV/6yledpK+86C98cFFgPm8Urc8GzwH2BR3TfJZyvGtfGDG+cxxf71TrqOKaZXk4Tsdxuk49582sLzr0BTnvgDbb78ptbX9eQ+eYhPKw34zFeCiP/f7v+Nmj/cRvWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yimdWwUgVRaL+g9yj0XDO6lquDp8/Q1nANrVdJT0HDOid9HPpSY0IdSalNK8un90JNCS+pmqtKG1VpWPM9Pt3rWOhGWAbRQ6VKG3U6VE8TwXs/XmZqqaq7lwNYh6oRjSi10UOPNpWqGALtTwHnjOZoGKBh1XtpoWGW61BDRh2RXif31ECVtNInWX+qnaSmjbpK7fe7fXlfp6Jdlbox1bAG2jOr1rnSalM/NX0eqSWnXq+oP3bAqaNw3lqh7Fn1yxxr1HDJ+Gj6sq4LzRvbc26eqPSt1OOqVrM8UvXe1VzJS870qVOyXq2L7U4KypltbPQ3CuW+6nkkeroVtIi7fanZ07mqw7GqMWQVspZUL3m/31A0zfRYrudwmZvYpjhUn08tnmNtc1yDyX7VS5vv2cdQuhCdN5+dp4JaYq0y6oP1dwaV7hw3oxr2BvM163PuZy/F73CqeWvm9wlz+uWI8ocH1C+rpn9GzxoR0a2kv7EPoQyqaa30zDFTngZzXpqumbF2mMNvWI0xxhhjzKLxgtUYY4wxxiyaeUlA5utrCTFVR+u+5siee8BYFV/hz9jVxEx5MiUL+rq6w77qdfr0eU4SwKt2uKaGKvj6vB8oETgevtPNyjKDVSL/0NH75YRs716W/3AxfWwahPIltNEj/EQrlDZr+BYhOYRTVHrQV7FVueae1h0IV0h/YCiaNlulwVR53k7CKR36EcPL2uZz4y4iopUyMUqkYfRaWnI8hNPvdzz6JKzWm2K77aawL0P55d/TZRsxlK8h7gESD9o9FaHmmfmGExfPo9+tQn2BQ7XNIB0p5DRsTxSikKTk+blA6RDSDL0mvkj5SqP9pD+PdORQDspHJHTJvlyEXTEvMzQt80Riu6FfDXIs7bFafY6g7iv7u1alWnPPlPJc7Od6bD03MUw9ba/XZV3S+qmcYzB+ChswyIooEVB7yzMlJaK8quvEGgrjuxVpRtvS3rDyQ5vOifmaMpNi/cNJuehD5S7242PXPxyMZ4/26xkr0QZtxLJrv2Hf7Chzk2N32/J5V/Qb2pNSv9Ko/aIlAcYYY4wx5prgBasxxhhjjFk0XrAaY4wxxphFM6thrVKVFXqQ4zYK1GBSQ6RWCT31elhD0/VBGVRPRR1Wd1wndD+3jaaw6DqeHq3S0eE8upuWR4mpykSbVv0VoXZNVZrJmTY6o63V5RYaVrn5wrooSo1zj/uBG1WMRR0z/eXxvjPgWD3v5a7UzOz3SJWpMsbjsqaIgE4Vfa6X7zaQ+6yD2iDR9DDNHqy0una6GcqU05yuu9I8SrrQM2lY19Cwqi65lrAWpnbFrjr1oGjdK6kp+oJMKl06PjdRl1rrXWfKztKJJ19baehVw4r7zGXbl5ZwnL6PWw1WWs1GNW4lVRpP6SftGd9x0FJH9c7Uks9ZRVXjQOqb44dWR71YzO135RhppC6oC0ypHGuFFrZhvzqeHjZj7OcZ676Msq/UdovlQ5ph1Ry2qJSiuA2fPyWj2irex07pmaLWl8p9V9OEjP37pGZNUp8dNMBz6Z8HWjx20vaYnCq7Pk2RW6UJLjdbtRBrOBfIfENd6kya3j3tDSuLQJkfB1qcqYYV1pEj+58calsrY4wxxhhzXfCC1RhjjDHGLJp5SUBlzyD78Jo5jkfvqvCNxmE6BNJpK6K2GXwNX77ChyQAcf9xJnsRQzJaXGazWklWiI7WFnMWFVXmJYRW9KK0bFFLHLxar8KfhQ7hfLZWPcLqfSeZfTpaeE0f96iXLWxzkvSVnCgXqeJ50/XRbnfFumpXRT0YHlNfGdiFMDQtoZihYehFQlOUctBuRWJXtLEasL3fS7gH0hzdrqx/YLWjFkl9/+TDMk+HOsw7hdpYtxpGZRasKstKESpFOBa6ola2G5ZHrW7ukwEqy9xVZfVhpqt8XPJRyqCOhxoPSH1h7DMbTyEBYZY4KTvlAgxvN5rdj/P+CelWN4ptDWsnpnWakwQwLqzDoHpWsQ6nz1vU03Y7SQTYVYb9cSlHJIxDyqAKG6nj1kYcAytIDXST18Djp+j3lNMV1ZeO9+vDbglp5/PMKZScqZ0ju0XXTZKkrvoerL6kvdN9dIRpxppJlTuUse33tIaS71XP9uqIJz7Rck331VnOKAnQtQfmYGYFld0rZKIr5rihlM+0fI7KZh6f/JziN6zGGGOMMWbReMFqjDHGGGMWjResxhhjjDFm0cxqWCk9VREF7T8KfdHxDIZX31XtRblzoDVLqCaGghTRkM2lWIyIJHoaarYq+64Zyxe1zKA2pNKtqa6FViYQn6rekVYmqg1pq5R6MzYUZ9SwMm2qVjH1rXrrtLGiq8Yo4qvMlJZox15Sow7Q2qludUilRUk/st00zR7SHVZWZtPNtLiXtaQI7NivoXnS4UQrtcr1Y1BbnvKindbJwMpkWk2p2/M40ETAfqURDWs1ZgsNK1NCHk9ZmmlFR12i9NUBKQOTeJG1KGtbpWNU2yBa6yAVbzo+j/XD1G+3sBer5ybRhlfFOd6InDeKtM3QOrOPq7Z4OGdq1hnbIdr56SbT2VIHnKRdx0rnDf2zbqDCNaXkCC3idlduqx6bY43zT5Fee8WUqqJh5jMY26383iJoY4XtpLpVPNeKFNj0AcPcpNrdyhbuRFT9RJ61mdptXXtwnYJ702danS79eOpvpiFV/eYI8fAaOlodw0NlaYa5XsY0Ncpa9jq9fHlsFJafTPEKTWtRJ9DCSnn3tNyDnlnXWE9lleI3rMYYY4wxZtF4wWqMMcYYYxaNF6zGGGOMMWbRzPuwclt1lkz1V6TvO+5rGhHRdirYgw6CXozqmTijdqhSz1GgIrqS+r6g/ZEyUbKjeg9qQ0Z6sal/IfSXDfRmeqZhzr+uFqDMHDrjC/sMw3RveZB6WkN32xz3XRuhmdkX2qAG+8rvbjWlITSFexHO7qEx6kf2OflcSYTLi66kD9BveC2p6xpojRto5TZSRWv4IAZS4I276T7XOLRZ6T9QNw0dkWiQEjXWJyIxhaBsJ/o/Ftq+suwdBuYgImWmxSy8VQNzTJX7+bi+frO5KA+d8bncXJTHahpFpiPWtMbsi6mF/lvrodL1Yt7Q3xx00KaJVo2Wpok+seoLms/ow9qV+r4yUy/mU7mJHfSje+huS59t6NdxbN9P5+J5BtmnnyMiLnfb8jwyqXCk9TOewi0moLabrnPjZulTu8Kcl2WMYOqMYUbvXGUk1hTE47zisCnKcKY5hRpN9bSmb7ZqMNvjevoI6Df5vMbcoGm5G85NOt7pCc3Bp37xPefr8ti1aJRTldJen5u4BH8rUHzxPp7MK9UA0zNf/OKrd6H8XYGu6548fsNqjDHGGGMWjResxhhjjDFm0cxKAvg6vUj1h7f9asGQEKcc8Yq8lVA1Q+OVXYm8Oh7jeFirWnvzNBKGZlh3qEIgEs7m63M9T+VswRPd+5wRES0tNIr0h8fLQxsrhv01zFvb8JyOhhZAEl7pVpti3yhlbFj5bXl/ahGygxXY7S3CcBJCHpD+cJen8uwQ1qJEYSc2XLTdYqOvV1OZVgi7buTYDS1ncNpe7GCYSnQFG65B0t41HIhShga2I2yjJCGcWRnKMwlT0kp6P4b21NIsIabJe1lLeHyPUNoKoWW1Rxrho9bKPtrWMexWhAGrfgK7Po3kV+lNJbSX2A7H5x+mdw7MDUnqhJIAPS1TulLCVdrsnc8qr4OUowy1wuJO5sGqKbCtqY4rSUBlOSY2TUyZLLHWPZ8pTDEuVdzRmhCSAL3mgLvR4TMwVeyINJ/9dNE1w7cYT+vCXg7yOv2M5q/rWizjmDf8RBT2XVHaxiXIZhqxCWvXsAxbHbexq9IXc20kEq8O40n7VGVVxZTdmg4WfarDGFY5AcdKX/TNck6hBLJYD8FirbI2leKy/60KGzX0t0qeKPcZTx6/YTXGGGOMMYvGC1ZjjDHGGLNovGA1xhhjjDGLZj41K7QOvfgIMX2o2s7Q+mJkqtEi+yDSpELfo1o1atzUIidXek2KbHUXrkl9hRSfOg3V/VHfWit6jmul+FW1RckzKTR5RdpuqTa2Sh17QthXVqoVouWV/J3Uo2+M0D2Noufc7cv63aKatsP03S10oJdyzUv0zz10RFqGPTS21BytReS6WZXnvZCufLGCvjWVusqV/u3YQ0dUNaOOEaaWnMqwwhdpBZVkUGx37LunYbUu9cxqXcSUqispb07Htc0REe1KteWc1spjO9HNVseKFo22dRy0qrGltlnTrUZEaHbipkqvrJdgHmvep6R5hI1Vk6nzFYucSt8qdUtrQaa21WMrG7DTQQu0MjUrtX9aLv7WgWlzp88jdJ/USqvNXuXoVKTGxPihRZteH+lWG2iudVxSf5tEx7iv0spiW54pCZZNeeQzULb5fFSrN9QtU6kXvYwWdieCton6XOZ9aopial9barnlM20bW84FvaZphr5V6mjfl/W3g/1Z0Qw4D3WznWjhObfrbXdcGzHFsBycOuqXoRWX39OwDtTWMY1lfVWWcPrcfwop5P2G1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiyaeZFJJS1QPQP84dTbLnNfqdFRzQRNUJkiUtO4UrNVym7gxUaNqJY9Uf+EbdGRDQP1RVI2+tVV51XzxXktZBLtXKWplbL30LdWiSXlOudKtxlR62VVt9pA/zOIFnqsskke1/flgdo6+u9N+/soNVvR3Zy+R401crxuRZfKtInUGLbiqDrAXbWXfr7DNZiqV/92bDFeeuiwLtrpXtZdWb62GWQfNKHw7Rz7SUy+z4Ww/GSwL5R+oEzbKhot1Pt+f3yMRIv7rFLxig6rPT4FMt1q5dkpc0O/Y0pP+HlK30jQziX16KX2cSb1LscOPW71uxw7KkXN+B69SHWeq3yyT0jlVytat8orVNMyM01lc1yHV907yqCtusOcvdvL+OHvNPibhcI/F++JMFXpY3nEfD/KmGY7VbpUKQP1kHw+lumC6Tkqn/lcxTSmml/+HuVUUIuq46nSsBbtUJ4nZ6ZBntFu4741TSnt2At9MyqMPtDqV1r5xaOP6ZzIlOZZ5ib+hqTydtbf7IysA35Vte+shKOXiDZxXjuuM57Db1iNMcYYY8yi8YLVGGOMMcYsmvu8s0e4UUPeeB1cvL1muLtKaSgpz7hk5mt6CdIMsP9Qe4ZhpCTguD0M06TmgXFp+Yj7bPSVPSxRmAq1W0toAvHFESHEXMgJIKEQi5wEe5qRIQZts/Z8f48wfKdh9xFWHhpeoR1W3iL9oYTS2+ZGsW+NlJtFULsrj921F1PZYIc1op9rmlKm0GWH1RAZJSKt9B22G8dI00xxwRUiOC36w2Y1nffGuizfLamSC6T5HPs75bYEPLuuikueBlqYSPvTRkrDtS3Ci2yzXuqIDkMJ42Ask02W19R+ixAcpRkh4bx+W0oqBtqhSVhugOWV2iOtmotiF612Bpkg06rs/x3uU+dO1m1bzIHzyUyLUOlTsKB5uuQqgbHuK8uoIXmGqStZj4xLhl13eBb0Mt/3uPe9bO4YAl2XbaN9OdM+CfZPGsGt5k5tR8y5PVM6S7/qK6uqKqb9xMcqzC/HdrRzChxbfPc8fYWpblU2U8kkCms0hNw5cagkAHsoJdHwPKUF424a72pfGRHRcK0k7TRW5UF592qFOZfiFaep1j+6pinLTtlJ0nTU1dwk4wpl7fhQk3TjlEXM4TesxhhjjDFm0XjBaowxxhhjFo0XrMYYY4wxZtHMaliRpauw1IA8rtCKjLnUaQx9qa9YbVS3BnsSWPEU2jToi9SqZTsixRk0WyvRFNV2T9CciAaFWp8sWkSmFaUOsJV7GwfYBtGuRHRtw748dtxN29TkZMpvm40ce550mxG1Dkt1qgMsfjT1aYL2eAVN60Z0Mqu21KVuVjeL7WYjaVyhBdxLKtQt6mxTys3illoAVWkroWHVFHjQBq2k77QYExewtboh1XCrK/dtoJW+pSlfO+hbxdaqi3JMDEyPJ/upAzsV1FKqzVWdlrST47ALYzhJHY17pNukzZFo0xLG06g6eejM97dLDfCwneqv35Z1zXcBqmkcqRcWW54V0x3Crm/TTOdZ4Ty0ttEUkQ0s4BqxOGugRGyoW9N5tznfnELJcJJ5g1aEe9Hh7fGc2O1nfi+AMbvDXLXT5xpTbkqK1Q7pVlcX5fzTS5ka6PlaekXpfpQ9iQ0bfzfAZ0MjGmemUK2E3jI38HcFRfGob4VutkgzfKY0vpy7VO9OjbJWQ/XbFaApacfK8orrAqlPtEMv6Vf3l3ePFaf67lj51pX0kg6WdoGq2x/Znpzrx+PrnYZ2aLK/SlWslmbYSb2rlpeWa3P4DasxxhhjjFk0XrAaY4wxxphFMxsHzMxYpa98udTVMDq+x1fvu0uxyYgy5NEwm5WEEEfYwaiNx76nHUlZvGHQuO/x+4ooQzQtX/2LDICV1yNbUBJLEg3rHwoEKyWxu0gIBTRSRwxhMovFei0hI1oynRBKAlKWUG+G7GOc9rWwrrm5LsP+F2JllVYPFfvyupQE3JQQyT6VIbq9hET3tMtpN8XmKKE2hrzukR5l2sUsZNJXaE110VASMJ33RoMwP8bIDdHjrOHRlfvHn/jcX2Ifsh3paRP1PyeiklSoJABygSQh7yr5G8aIJqxqIanY7o9n2qOlShYZQP94Gb67+5bHi+3dndtPfB4gCWBmnPZi6sfNugwXdzen7Q3CdXOyrFiXfZzZwDR+S4e7Tuqa00RLSYD0R9rznRK2m8oY+KpFZQD78XhYPyJi0KyMOM8lQuWqZqOV0SiVyvOMyL6kkpBujbmStnp7kcJ09GiTC2FXwwCzlgFzMK2r9vJ8qhJxtSItoOwE86w+Ls8lSaPlncqORq5hjm7UfVstpoZdufbgfKQyoxH9VmUA+7vlnMJrDpLBj7KDNcZ7EukIZY7Fo4gyrCpDpzzD7vO4a2T+yTOPjEr2gu1OJmy6/M3hN6zGGGOMMWbReMFqjDHGGGMWjResxhhjjDFm0TwlDWuRhhLiKtWFVZYzEFD146SX6ZmitC+1YKpj62Et0ct6m3KZEXrCrWhHaLFQpXaL41rdQmM7Un8CKwcRA1HHO2yhiRENbsKxelamX6TubyWCtFzlZDsdLdJEjjvpD7BfWYmedNWU+tGIcrvrJt3q5tY7FPuGVal3vSXdeYCNz040tdsG2r8bt4rtLP2KWi/qsrRtRtiRZbEzWWNMrFN5nk70Smukcb2BUdqJVrqFndsgY2uAVjfDIqnvVS91Pm2iUuqnYJEjtTtgLqosj/S7EOFVaVxF/7qHhnV/Z7Ku2j1eala3dy6L7TuPTXNKf1m2A9OBdhuxG7tRju+13Mwt6NTWGO+DWlWt8VuBltrImfSMWkd0PELZG00Hi7F8Sug6pNZQGcLbQco1QNffU3uq6TkxDse2rP9B+s6YKBqVtsAzJcNyTG2vaOPDfp8a/c0CylfIeI9b7EWUtlYZExmtJlUv2cP+TsWKGWOrZZ3osRykpwLPaG1u/sZD1zS03RppgaUpSzPrD2uRndhvwlZP9a987vfUxvZaPmixmf5Z1mCrTamL198kVDJjWCGqlWiVXh5ro2HGLmvUuZzngaZbLQzT8OTfm/oNqzHGGGOMWTResBpjjDHGmEXjBasxxhhjjFk0sxrWBsKnVnRD65ZfFa0ItBeQkcROZC899Ho7eJiFpLSk9kc1ePRdZUqxQfRPtBLscinyaFT/wTV9Pq4xoWZnFN3kSH+/vtSjNIV/HVJoyn220FhRx9TvJ11dpnjllECHpTq4KsVcnrRVLfxb19DCXrSTNucCmtWhK/WuN/V+sW8rXX0Pr9fmAilepW/TX3a/K7WKhVYa2q8smtYGqXkT0q22akYHDdQIfeRevqua1cN55XOmfotiL03zeC6983GdKseaDif64TLVpEry9pWsrqwH9Wa8hKbs7u1p/Gxvl56Jexx7KXNV33N8l32hjamd1h303tLeDdo6IW9wWks6aujo6NG8Xsk8xt8cyGfqg5maVY8emZvzhFA/rrc3MDWrlHmH9u/Zr+S7Gf1opA5Y6qsFx4kAACAASURBVK1D2nA1t6VnZsB3Mq2OP2r5O4RG0lUP8O/WZxPTcrPZqGkurkmv9OLLeI6J3nConodlnai/ZmYBT0THtJ/a15laVDOozuh4IyIGTVmK59uAsbYXLWrGuBxl354pnHFsI52+h75+xBydpf81GCzdjem5ST1p4L71vSV/f8S61W1qWFVXzmuuoM3X+mNK6Tn8htUYY4wxxiwaL1iNMcYYY8yimZUEjAhbRxHKL3dpWsoGYUqGATt5BdyPTM3KMkz7GYxSu4gdwqh8la2WWANC1D1eXw/y+nqzKUPLraanpV0KwiyDhmCrVG6w/pLzMq1ZJ+Wr08iW59Xw+xmzKFYp3YoIE1KCNhqKHrGPXVJDTghldKjDoskRstEmvrkqwxMtwhUrSZ1ZtfGq3N5txeoI58n9VKD97eM2ZhEReT+FivZ3SjulsS/tlNQijaG9KOQC6I+wD2m76T5XF+eRj1RON0/SJil1PA7hKOknGf2ENk1Z5Es9QmmqSOrRF+/CtmwvfTdTqsNtDQm3x+10aIdFiZSG/sagHRLmDUkBiqhzkQ6UKT072otpusgzzinsK3q/DMmqRITyEaZULYLfvFdGJ6Xe2s2MdAyhVKYL1ctwrmzQOGqd12J8qISqTuuJOUXs+2hXNDBcL8+NOsHvVIYdxlbLMLV0EFppnYqGVo8iA2B6cg1V0wqTbT+qr9oOawasN8o2PW6BNcDGM3rIv9RqcOQaofyqWnYNsLzS/pfQF/nsUc+76pq4F1VY0PJT52hKkGiDKtnFY2yevHTEb1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMopnVsO5hqaFSm4bp5GbETdxX2Bwh/16lehGhRAuLipXohnbQze4vy2tuRY9Lu40BGqJhN23nHhZIanlEJxPo7FZiX8OUhg01HYWtFXQjcl6mOeW22pNQu3dKNpuynva3J90ltZM70QJSI6o2ZhERWVKPNlHaDK1u4LutaHFo+bOWVLHbO8W+Br1uv5XrUP+GtlENK9P3hVpZ7cprxrbUpfa3H5NDHyuPhSWWWqTRLm2QY/d9ec0hlfqpjeR87TZMkXsiEvVmYhsES7OVbHf0xoN31RhTv0k7zillu6hulbZlIdt72IBVqU9VC810ptCbqUaYqapVR0n95VhpykQnxrmA9khSZw20fHpkBx1iGjAGNR31GVOzUqOr9UQ7quK4YB+jLl5SvDJlMvpgEs0608HqHL5al9doabUk3+Wzk9uF/J96Q9XxQg+5D+gh1RaOdcJ70ZSqtHsaxdaK2mE+S4t0wE/erujpUGk7te6rIsjcyR+AcNDKeMotUnLjUJ3WKoszeS5UWmL+BEHTwXJOoRD9uBNZ2fhVenHo4uVmWD6uafSHIsyoWliuoS6r3ybJmnBkDuYZ/IbVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLJp5H1akj9S0a/QDVf2Z+mxGRATSr4ZoYsYBaUihycuir6J+Zica2wG6qx66CN0/YN+eHq5yn9SU6T5KXm41t8pjRe9Fz7dxz9SN+j1qsORzeZpKw7gX3cu+0j+dELS5WgJeXpb1u9tO9z7Ah+7u3bLMF5tpf78v2/8GtVai2VMPwoiI9ob4+SI9Xp/fXGwXno73kWGpxiyP9P+Ue9uVmlVuq262vyy1pz2O3cr2gBSgvepdmWYPMlVNCXkzXcQ5qGxY1WMY+qlONKJth9R+qewnW+lTDfSuKcpjVRdKnaSmoaRHZ6J/r3xmOtNKJybar45+vYXWFGMf5VMdf6WZh8atk+0W512JhnFF/8ktx5WkbnwKaRSfLkx3q/q5HvW9l7E3YNBWKSRlN7NWsg+uN9O4oCSzEX3r5qIcXN3quIaVOkG2ser6q5Te8qy6O5TzQiVxVD1k5QPOlJvT83vE81qfj9WYoM47tOznS+OrFH2dfVuKxN/E0P9cfhJR+SO3F6XWOcsDb4D/bJbx3qEdtjC01+bew/d5xHyu6yGmote+Sn11s0bf3Ez71zfKfnyB7VFTs/J3OLqP6Z4xJ+tvQZ5KWnC/YTXGGGOMMYvGC1ZjjDHGGLNoZiUBGWH2JK/MO8QfVvKambZRDKu3Sa2h8Dp4V4ZrNay9xb67Yg3Ujwz7IawhoWemY6StQiOpwnq8st+I9GGDEOGA9/1Dq3YWZVV3eJ2+kfRplARoHIPVxTiVZjljKPSUjAgN9XnqD3eRMvCOWF7dvYtQS1u28Y3NdCylBd1bYP+kFknrMsR98dBznvicYOG0R+hKm5z9ijKQYVB7jrLPZbFSo1VQfxeSAEnNOiAUtL17u9h+y+/93rTBCK2EV5qu3HnzVtlfOwl3PpLOlJqVocnCfaW6Gd1Z0M5YA7UYP5yrVM5UqWY0PI+Uhg0ObnT8o590GO8bSevJVMCDlHeFOWW14rwxlW8Nmy3ORxdrsQXDTK8pNTvkjR5pyTRMXx4epCRA0++29OqRdJxMUQsLJ03TzamW4Xp9VjEPtso3NphvNpsyZKwh2w7nYeRco6kMrapGab1COlg8zgeRnWU8x/gYKdJlcy5vpjJ0tFHEefTRP2c99kxStdmMJCDJfN2gnydM7ioP4tgf8SBu5Ng955SiH8NCEwdv9Tq03cJ41/LVEs3pGcc0wS0kAiuRN1AWwTrR+SfzWCkv7boodWqa6Zo97FPn8BtWY4wxxhizaLxgNcYYY4wxi8YLVmOMMcYYs2juo2GF9VKahAm0hFBNWYJmIsFiaBB9adrTlqfUMF7uRHtKsY9olRLUNGqdFBGxlfNQX1GnoNXUbtRDyXWYtpMWOXKdNXRrG9j0rGS7SkOpqThhY1TZlUhq092u1E2ektWq1HDlZqrTS9ivPLadyvXYnbLumyi3b8ixO1qLsJ5EV6QanoiI7rG3TN/boKzQOO7GQghckKBdGkUbndEYSdqNVlp3bpfWVTs5zwirr8u75bGPPT5pd5lStRMt+Qq587ob0J+1mpr1wdhaqfawwXjS9q2kpjhTJzo26lupYb2Qe91dlONJx1eGDrGFTjXLsUwxvFmX31Xbo1SlO5y+e3GzTHG8ulG2y60b0/4N9K0rzLtr0bSu4V2VQu4TOjqmH9Z0kbRZOiWXSD3adKLPxrsW1RtvaLezQr+S5w81opynu3babqgnFg3fpivb6Qbmn3YmNSufI2qhRK1kEkvILvgbirIddzHNuyNs4Ea+qlLtKfTXOl4yfv+xx5jYSXlnsrU/o1CjXGzCMkn1rVUK1erE8hFzwZ7SYtGBrzcYa6pJRnrsDmmkd/K8o+A/0w5NLTbZNzeqS6WNXll2HUod9a5c57W6zsP6R45tMedSR9vLnLPmfDiD37AaY4wxxphF4wWrMcYYY4xZNLOSgKZ6nS7WHLQ+aY6/as/B0LnYRSD80LZ8JS2WKghVdWsJsw0oD7JXNfIKekT5Vqvydf9KzrvZlCG6RsuH7E6078oa7sR9rhGCVUkAEz90GkJalWGC3WUZat5l2c7ns7ViJqKthEvfcnm32HdHLJwue2TzQDarrYTSd5BD0ONL62l9gbD/HZEEIOVTjxDdTmzO2JcTZDKd2kjR80osSvaQZ1xuL3GoZErZ89jt0e0WdlmbPLXDBsP7VkJf1gwsa6TBOhED7HU0/NlB8qGh6IR5YdWW/e2GhOHgaBcxln0sXahdFrLTiQaknBUiLhHjHLYyH+JP/65l+FizycACSbLJMNPMBsdq5pkLhClXCBkWVl9VdiWZL1uG9orN6OW+B+qpTsgeUgW1++s2kJ21x++nxbOqTVJvbDeEVltp2Aah540+J7qyLTo8G9bSX2ldRt3R5f64rEuTFGVY0fUBG0oZE3tIrQZYmWkJ+BxLIgNg+1fqBpkPxzM9f9rE58BMBig5tLrPGXuxvrJ3wtJJ+xEj3HKeHbKwtZAOtSLx2sNukwIGlXUkWEypGqxHWzeVzk0t4cp9tNXTsURbsKI8zC5XXjFGtdXj5DmD37AaY4wxxphF4wWrMcYYY4xZNF6wGmOMMcaYRTOrYQ3qXNK9P0dEkbOygd3QCjoN1bbQCorWDZqOLF3ewb7pu0Mu9TxMVXZxcfOJz3vo6GjVojoNWidtRGPC1IgddHVq5VClfat0vnIeaEPUsidXOc9KfeOoNiOVmO90VLZIcrs9tEyD+HzkVN7Pdii1VnelzXfQa9bpOqfPq13Z59RqrYHebIs6vSvWIhm2LZuqbaZCrKjFke/2O1xjd7zd+srqDfUnHjQj+4NaATW0YinPq+NSx8cp6ZHiUNNvjrRTks0q3Srs5lbS+BcUn0KU2bWSmjOo1xN7GmikN5gL+u3UVxv08WoMi83R5lZZ16phVVuyiIh2xbKLjhOayhXmmCLtNe2R5DPthxpq8WX/OTWs0VCnutKNYp/+BmBEPSS8l1FdHrV2K2gBS8uf8tgLmUfWLe2wqLmWNLm8LzyrRtGM5gaab22bjPm9+vGD7irLPkDvWjyTac9XiTJlH1Mka3mfgjbx6VCNPdEP89meZZt3xTWN2jSNTJMKGXJhs8ZVlYw9usL1+F1Gu5PfFYylnj0zL7v+puhm+fwtNK0oe4LllK7BBtYXLylzYIvz9vpd2j/iOVrYQfK5PoPfsBpjjDHGmEXjBasxxhhjjFk0XrAaY4wxxphFM6thHeGR2RfbpQ5rLV6m1KwmaB12++k89O/cQEiyH1XPA21S4flWai828FYdRaexhY6OuixNR0YNq94ndUqVzk50k+s1vV7h26dpMqlxErHmAM0LNTB78Tjt0X6nZH1RenxuZJveuv046VL3IzxHkZp3L56Ed6F12TN1o2iu6ZOnGtsGnqOVDHSv+pqyr9yE9+XFerrOnj6i0s+227Lsjz1+uzxWfVip/4FYV9PeUW+41r7D/IvQPKqPXru6j5z9GYIa1kKvR7M+2eZ99hSDqXQO4/ACc0GraUl7jH3Rk/Y4z4B+01/K+ELfZCpC9bldQ2S72kgK1RsQ4NLbN+mumZSUEVHIX5HuV9NjB9qEnsGD1NEwsJFOR5Xqurg0tZOi7YV+mCkkOR8Vx0JHrXW8ppe2aFgrH9aWWvepjfmcoL6vFX3umkWV+t+jLZiSOEl5qUvN+XjdMkVyko419nwG83cu6tl6ptTg9HuVe+Xw0cc5/Ug7rEX09wF81vcd7ls914P9bdrmM2LNdpGJrEPj7/Gsb+S8K2hYVzLHrJlimt6qctuVvyx00dq+VT9O+nsEtEmlYZ3mTmtYjTHGGGPMtcELVmOMMcYYs2juEwdE6FxC+SMscsoUaAhpMuYqoQu+hs8Ia2w203eZ0k7LMCBlJs+jKVXvIuTFTGUaUmyY0lDTA8JapYP0oZBJ4FU776VTuwiGdqQ6B4YQ8Dp9lO3hjJKAzc0yxeVKUkpWaeMkdD8ipNRnhP3FuiUhDLyHBdZuN21XdkV3pxD86kYpZ9lX0Yvpml0VUyoP3vbH+73KGe7cKVOx3r1bpqvVkAmtdhjXKlIkZ1iUtJK6s0Uq1o6hviKXXpyD7d2yHlQ6sr4s23MjVlAj/ram9dcgKQ8533SQ9TQSrkWEM1oZl0wF3dOKTlN8jrDVY7hRwv6V5ZW0S2VNhdDyIOHPxDAbUvoWXSNTiqHzRFmXW7SD9uNMKcYJofhAnzEt6qUoFq14+F5G+gqtwRK+XEhRKvuv488CXlPDz1WaXIbnk0hWcNGNxG9HzJWU1OhZaWtVyc5kLujxLB3lxJXtF+prEInXMJ7n+TOivL2sU6oU8tLefM7SuUr3ch1QpXwdJL0pJRWSQne1RcrSkSnup+fmaoOxj3GqPWNzswz73xBp0+ailDI1aMOmm8q+WnNugrymKD6lGLJdrVNQdr0X+urN4DesxhhjjDFm0XjBaowxxhhjFo0XrMYYY4wxZtHcx9aKGpn+6L5eUk1Sa5Gg39Q0pQ1sExpa+Ohn6I1U15ZH6vOOp4ftKi3scSuTjH2q4aJmdQW9x1rthwLALknlNFUKNt2GDRTTrzaaru+MqVk3F6WGdXPzpnwuLTdyK3Zk0IQyFWFaa1paagqhXQpNL1iWrxd9awvt31BZvExf3qCvDHdpp3Q8HV0vY+RyC4s4aI81/W4DTU8LfZz27baDDkv0kLRF6dblvaxF29SumM/0NPQ7zhvT9v4u6miltlGwDMO2DiemsqWlXKk9hAXNSrV85Xl2mNeyjP/MtJg4r/aj9ea4Lmzcl3UwDNBYFto59FtMMiphT9SJabpIWHvt0VeHvc770K2dkGoalLmY9mi5mdGINpzfdR80w9Qf6meUR+uihxCelkkhWv10HzvEQeanDk8Okd/Gui37dZ9ntKc4dkx4JuuzvcowLbrpyjoL2mj9vcVIFfJpYPrsQp9NnaXU53oDbSc16tJvqFllHxuSXAcdRW2tHnq4bIcdVmCawrvfww50B92sfL75UPmMvRBbK+rg1xf4/YzoVvlThsz1mKb7xX3q+pBad2pYs6xNRttaGWOMMcaY64IXrMYYY4wxZtF4wWqMMcYYYxbNrIZ1tyu9IldbSUO5K1NobmVfA81qk0vNhHq5UTdCm7dc5BeEz6XqsnjNGd9TKmsotVGlyDDQa0zOyWvS+1V0L0z5N8AzsZMTM42eenT2qPdhB92fbPe78/mwJvj6XYjXKbVCqsG8hJ/mLtNPV3SC6Bt7aGh66Sps4/1+qtPtrvQC7amNlfreDWXZW/qyqlaH2ljZN2AfvUFTd1ybuEIqz7VojipPz83UDpsb0DXdLH1ZL25M292qvM9TMaKuR2mXHql2L29P8896A5/TSmwlunjKNffH9ZsthJIqd6Tj5Bq6tUJajh5HrZ9IsWOF86gPa5+P96GIiCQXbaBD5ETWixY14T5VQ1ale97Rf/vBaFipBVQP6nGm3Tj4K/muaHap0UsrajvTPT9HlNpy7qMPb5ECuJ154ESZ2pMpXkfRZI4dyh707ZRjcY2A/r/R73Kilb48whe2mjt7/R3BeTSs1NGWFYp+r9r3zDSk9FrVfZi34B/eSFs0rL4i7W05l2+gZ99eiqYfmtV2Cx2/3OfFLayx1E+VfrPloXFDvlun8EXaYO0m6AtJdLyVPzx/TyMewiPSsc/hN6zGGGOMMWbReMFqjDHGGGMWzXxqVrzG1VSfu+2dYp++TW/XZXixGxlWl5AmrKFowaDZJBl+UGuJlinYEK7vZJth/sp9Q16htyifpoFr5iM7Rbh4HOdteXZS17Sg0ZDd9k5Z73fv3C62Ly+ncPd2++RftT9dRoaxpQ/cQGh6IxKBDiG41ViGTLKE3QbU2SaVKVaThC4Zgh+2U/kG2PiwIRuNnyGd6YAw16ipMmnJJX1nDcnETaTLW0s9UGqyhh2V9skV6u+GhP1vPeehct9D5fZa0qKuNueRBDA0rTIKphbtNY0iz1Olr53qoaVlT8/xNG1zDKt9VhX+ZgZBCQMzreMK/6BWanlLSzuxF8uUFjC+LZ97WtwdD+X2lV2Whh6R4hjpc3tpF1penRI2TbGdaBt1PM3wnFUVbQv5MNDQfoP0ppq2FRHiGCCrUC+tPsp2S0wdLH2b0pe2sEqkXITWQSpXKotTuT2ppAZzlW4xtJtxTa1bWj+dCqq09Gb53FUbs4G2nVVKVbG3TLAMQ33qOoZjVmVktL7sIQ8prMCoeqIkST6vLpAKWiRmtG5rO9gmtsdlHJU8Ua46Yh1VWFexL86kkOexc/gNqzHGGGOMWTResBpjjDHGmEXjBasxxhhjjFk0sxpWptdS3dNuW+qc1AOio+6B6VebSZPXQxeUkMKLmsHiWFlvU8fClXgzdx6UTzUf1ddEtsEUmjx4LGyNYBcBrdROdGID9GaaBvcS+rLLu6X1mKYDpSXFKYGrS6G7fOihMm3rI488/MTnLfRzzZ3yfgp9FzRGW2ge74p+985lWU+DWrMgPWiCNlrbhvKopqNOUNMWlrsuRKd6gfSgG2zfFJ3vzRulNpca1o1sU2O9kWs+8uijxb5H3uGRYvvGraldug5eJ6eCtimqI4MeN4aprqmdrGytpB2aBnqp/rgOlPpGHbMjLNf2W/Qb6R0rpI2u7nMmHeioWnxo+GmjNmgKTVqEpePjnakR1ZaQcznrent5PO32KRkxqRTaSu4TrXsLDSbrVHXUHLMt7eaKNOLH07hyHzWjhfQYmmrq0DXN+B4pX7fSNtQeU+O/32lfwbMcVm/6nJ2RLRbP3Ijavq3U31bi0pMwUB+Zj+s3VWzOrszfDuQyMW+xr55/jttSFqm0uTBZwdZzLf2N/RZ9U63duo59Ss9b7uPvhvI49SPaxVW2YLLZowKHYSefofkdOHfK7wio05/Bb1iNMcYYY8yi8YLVGGOMMcYsmllJwAhrILVG2e9gHVNYLsxbs6jFAe1pmjkrDLyGVxsKBh8YjtCMJomGKQiX5KznZahHwiy4ZhV222s2B9hXMLuM1G2PULdmxdoifLeFJGAvVlZ8DX9K2G4amn4O7JXe9V3f5YnPLULRNx97vNgeJLzDwAFtux67PZ2rskeTcE+/gR0HQjiarYX9SjOmRUSM4xTKp42LSgJu3iizTN2ArdVDEp7XzxERK8gH1AKLIWOt90cfKSUA7wRJwK1bx8t+KjKsUHQc9JBqrCS0O1ZhI4YmB90o91Wp7eQjh7eMU4ZuR4zZtpiPyvL0PfvYtM0QdRF6hLtYZZckc0xP+7ugJGm6Jm3edI7p97QvRKy0sEc6T/aiiKjC/prJiWH/Yp6mNRXOo3NBwiyucrWIMtMUx0gnZWDmOoZWtU4Zpu5gnZelvHuE7ndbsTi8xHMCY0tDtpSg0V5uHI9LAvReOFdSClE8z88lCYCELotXXSUHkT40YHCxL+jw5vOkaZklS2QIleWalBXjsEFfUElA7OmbyTlatRqQAxVylaNFPZxFM/ZRkkn/LrkZ7lObsB0zcHK9I321yiY6g9+wGmOMMcaYReMFqzHGGGOMWTResBpjjDHGmEUzq2Htq9Rl0+e2gx2RCiWoCYTGRPUgCRYLK2gai2MryxHZrjI+Mh1sHIX2XVlFHtD6FHVCt5xK0yFpJ0fWSVm3alFCTY5ecwfd5p5610F1s+eztSLrzdSODz9calh3Uq41tJ2PwLZrJ7YuvNfLy7IuHnp4+u5d2Fpdqra3alOm3ZvaihrrFtol7fYdbFHUumqD1Ke3kK72htTDjYtyH21v1mKttVoz5et0nps3S3ush5GaVa/T0JbpRLBP9mK9kzJSCYvAtO2oWcRcILZG1FNTZ6cWP/yLXfdp2Q77oPvTz6i/DtWp+i7OeXqeKp0yBbg6NfG+qGHNqmHF/CP3uavukzY4x+fZU8Kxp8OWU1srLZmhPOe9N2IbRRsrqtaLVKi0I9O+gnrZ09ZHCk+5+Ih+r/ZdtLVS66oRulTep+6nTvkev/qYjkX7a9n3OM92e1yrSGujU1E5TOk6BfuKbVrPQV9apGbls54pTFUbS49HqU9qhzn/qJ5zGNmHqLnVDaQQl1vp0S+omdYmHSrtPfqCXJX9T9cpTHtLnWq5bVsrY4wxxhhzTfCC1RhjjDHGLBovWI0xxhhjzKKZ92EdqGeYtAY7+pzqPqa+hE6oFQ0e/c22OK/6dFI/qCnPeB565s2lieO+QbQZ9KItPAprYUt5rGhXqAWp0uqJkGSE/mPfH/dMpG+a6nColzkllUumtPkFtJTvIPtuPPRwse/ysqwX1a1W+inUoW7vqBFW30n06z00rMW9UI8ENDUqNazrlWhYZ1KxHvZPGtdKs4rtzWY6V4d+r99dr5jSlSafU9mZ1vFUUB+pfqXUiPZ79S4t70U1qxHlHDOXQjMiIou/Kls3S/kGaLTG/rgOnd7OXcv5UcYl5qaVeCtzHmMZupXspxay3Cy8p6lv1TFADTe1faqdq30Zz4fezw7zXtp3xZHF9+hXK21c/WaCfUX3od3Up7e+BvSQon+t+nLD3xqID+ucfyV9gqtU6lI++vmyb2ftDwP2TcduMT8zPbaOb471U0Ftsd7Z3HOfdVClaJe5nT62NEhW71yuGbLMTZwn+LuXvaydKvtonldTTGONtQ/9/Qy8X/vjdVJpVulRf3QDut6q3jkni5+rU7MaY4wxxpjrghesxhhjjDFm0cxKAiKOSwIqywp15qjStTGtma6Tj6cxiyjDrE1zPB0ar1mlUpNwTvWyGqEeDRVUdgwalqbNBN0sipAMw52QBGgavYwQUZEaj2FKhGRUEhDng+XS+r+AdVUjYbgLhohvIb2tpu5EqIqhTJVOVO1WhF4gAalsrtQ6pixfC5+Uol9Vcpbj4XmGIudSQK66si93RWpW9HMpA8/DsJHaklTpik8FJBYa9h9z2Z7FvbSQbeTj7cD7zpAaaNhtpI3LcDx0z5i7ynGq0BnCcK1IBBpIRwqJVEepA1Lvrqd+kyA7YNRylALTyk3Py7AkJw4NE57TKo9pLPfaNlWK2ClFdR5Rv5DUqLRsxDNuxM130ndoB6SWV5wnqmdDMb4Qoq3mlOm8vEt9plDGs2cof9RjOXcelw/wXrTvzFlnHfbLs+pcqVlxLzouKilEPi6T6OFN1qj9HcZai21N68pn/dhq2lu0EdvlqViByWUyczhLP+ZziXNeOvI5ou4LqVhHcaKY9vW0AMx8N6oSiiefQt5vWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yiSdQeGmOMMcYYsyT8htUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNOdj5lwAAIABJREFUMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsGi9YjTHGGGPMovGC1RhjjDHGLBovWI0xxhhjzKLxgtUYY4wxxiwaL1iNMcYYY8yi8YLVGGOMMcYsmrerBWtK6R+klL7qQZfDLIuU0nullH4mpfRYSukLH3R5zIMnpfSalNKfeNDlMNeHlNJLU0rfPrP/51NKH3nGIplrRkopp5Te80GX41R0D7oAxiyAL46I/yPn/AEPuiDGmLdPcs7v86DLYE5PSuk1EfG5OecfftBlebbxdvWG1ZgjvDAifv5eO1JK7ZnLYq4JKSW/EDDGPGk8Z8xzrResKaUPSCn99FWo97si4kL2fV5K6ZdSSr+TUvqBlNJzZd/HppRenVL6vZTS304p/Z8ppc99IDdhTkpK6Uci4qMi4htTSo+nlF6RUvofUko/mFK6HREflVJ6JKX0P6aUfiul9NqU0ktSSs3V99uU0stTSm9KKf1qSukLrsIynnie/bx/SunnruaB70opXUTcd+7IKaXPTyn9YkT8Yjrw9Sml37w6z8+llN736thNSulrU0qvSyn9Rkrp76SUbjygezXPICmlL0kpveHq2fPqlNLHXO1aX80lj11JAP4t+c4TMpQr+cArr/rdY1fPsX/zgdyMecZIKX1bRLwgIl519bz54qs54z9MKb0uIn4kpfSRKaVfw/e0b7QppS9PKf3yVd/4qZTS8+9xrT+eUnp9SumjznJzZ+DaLlhTSuuI+L6I+LaIeMeI+J6I+JSrfR8dES+LiE+NiHeLiNdGxHde7XvniHhlRHxZRLxTRLw6Iv7tMxffnImc80dHxI9GxBfknB+KiF1EfHpEfHVEPBwRPxYR/11EPBIRfyAiPiIiPisiPufqFJ8XER8fEe8fER8YEZ98zvKbk/KpEfHvRcS/HhHvFxGfPTd3CJ8cER8SEe8dER8bER8eEX8oIh6NiD8bEb99ddzfuPr394+I94yI50XEf3W62zHnIKX0XhHxBRHxQTnnhyPi4yLiNVe7/2Qc+sujEfEDEfGNM6f6U3F4br1jRLwiIr4vpbQ6UbHNGcg5f2ZEvC4iPunqefPdV7s+IiL+cBz6yv34KxHx5yLiEyLiORHxFyPijh6QUvq4iPhHEfEpOef//Zkp/YPn2i5YI+JDI2IVEX8z57zPOb8yIn7yat9nRMS35px/Oue8jcPi9I+llH5/HDrBz+ecvzfn3EfEN0TEvzp76c2D5Ptzzv805zxGxD4Oi4wvyzk/lnN+TUS8PCI+8+rYT42Iv5Vz/rWc85sj4q8/kBKbU/ANOedfzzn/TkS8Kg4Ly7m54628LOf8Oznnu3HoPw9HxL8RESnn/C9zzm9MKaU4/LHzn18d+1hE/DcR8WlnuztzKoaI2ETEe6eUVjnn1+Scf/lq34/lnH8w5zzE4WXK3FvTn8o5vzLnvI+Ir4tDhPBDT1py86B4ac759tWccT8+NyJeknN+dT7wsznn35b9fyYivikiPiHn/BMnKe0D4jovWJ8bEW/IOWf5t9fKvrd+jpzz43F46/G8q32vl305IorX8+ba83r5/M4RsQ7pL1efn3f1+bk4Xj+bZzf6h+qdiHgo5ueOt6Lzx4/E4S3afx8Rv5FS+qaU0nMi4l0i4mZE/FRK6XdTSr8bEf/L1b+bZzE551+KiBdHxEsj4jdTSt8pshH2qYsZ+ZD2ozEOz6HnHjnWPLt5Ks+N50fEL8/sf3FEfHfO+f95ekVaHtd5wfrGiHje1ZuMt/KCq///ehx+aBMRESmlW3EI/7/h6nvvLvuSbpu3C/SPnDfF4S3ZC+XfXhCHvhKB/hKHycRcX+bmjrei/Sdyzt+Qc/6jEfE+cZAAfFEc+tXdiHifnPOjV/89chUmNM9ycs6vyDn/8Tj0lRwH+cdT5Ym55Eoz/+5x6H/m2U2+z7/djsMfsxHxxA9/9Q/Z10fEe8yc/89ExCenlF78dAq5RK7zgvXHI6KPiC9MKXUppRdFxAdf7XtFRHxOSun9U0qbOITi/vlVuPefRMQfSSl98tVfvp8fEf/a+YtvlsBV6O67I+KrU0oPp5ReGAcN0Vv9FL87Iv6zlNLzUkqPRsSXPKCimvMwN3dUpJQ+KKX0IVfaw9sRcRkRw9Ubs2+OiK9PKb3r1bHPu9KemWcx6eDr/NFX/eMyDn+YDG/Dqf5oSulFV8+hF0fENiL+2TNYVPNg+I04/B7iGP9fHN68f+LVvPGSOEhM3srfi4j/OqX0B69+1Pl+KaV3kv2/HhEfE4e1z196pgv/ILm2C9ac8y4iXhQRnx0Rb46DDvF7r/b9bxHxVyPiH8fhDdl7xJV2LOf8pjj8hfI1cQj1vXdE/Is4TBbm7ZO/HIfFxq/E4UdYr4iIb73a980R8UMR8XMR8TMR8YNx+EPpbXlAmYUzN3cc4Tlx6CNvjoOU4Lcj4muv9n1JRPxSRPyzlNJbIuKHI+K9TlNyc0Y2cdCyvykOEoB3jYgvfxvO8/1xeG69OQ6a+Rdd6VnNs5uXRcRLrmRAf5o7c86/FxF/KQ4L0zfE4dmjssSvi8OLkh+KiLdExLdExA2c43VxWLR+SbpGDkeplHgachWK+bWI+Izr9Gs7cxpSSh8fEX8n5/zC+x5sjDH3IKX00oh4z5zzn3/QZTFmKVzbN6xPh5TSx6WUHr0K6Xx5RKRwKMbcg5TSjZTSJ1zJTp4XEV8REf/Tgy6XMcYYc53wgvXe/LE4/ArvTRHxSRHxyU/SbsK8/ZEi4ivjELb7mYj4l2EvTWOMMeYZxZIAY4wxxhizaPyG1RhjjDHGLJrZfOef/mHvVfoJjtNmk8byRG0r+/DWFpurbj19Xm2KfU1TrqF1q0nlPrVYbbEvowxZCpEjFfvKrYi2mVnHy8HjiGuM5Q/De9k/DDg2l/W32+2m7/XleYqyo7D1vUzbI96ef9uP/gJv9Rnjb3zphxcXG4fpHvJY3mtRLtTDOJTbw6B1URa/68oshaNcJ6PTjdI2I9qilb4bUbZN3/fYV5ZBi9915Xm07CPvE3XSyGlTg2ZCf9T+2TRsfzk2YR+O1XZoUAdf+bd+4iR95eXf8n1F5e/3x+tIy9/gXpq2rJN+nGuz8rxJ5gp0t2ilDRP6UL8vz8s5p7gG6lrnoxHzhLZn4n2i7bXf8BpdV07nXavbaHs5D+t2vS77gpZhhWv8R3/63z3ZnPIV3/LzRQMMOmZm6olRwyqGqN/NnCfKDqHnYtvovMa5lg8V/e79KkxPxfKU8+X8mJgrO7+rzw0WUOfHqn9iu5UydF1Zni//9D94kr7yqp/cls+emXbRfeWzpeoKcXA0fOvGvOlL+bwbju6rWqFhP5Z1VHN8vXPYvvf3Isq5tOqaOLaVeYLPao6eYs2FuSDNzGM8r/YpPvM/6UNuHe0nfsNqjDHGGGMWjResxhhjjDFm0XjBaowxxhhjFs2shnW9Whfbqr1qoRHtRLtCAUIHfZzqVlXPGlHrR1X/sWrL4q5Eb0ZtFUmtaIig4aD2q9CDUNMkmo49tHL7fZmEZCeaNx47QP+q+iOep9CjUHtU6W2THHs+VugrfZ40uZlaHNXhUQfYQDeYjt8P9Ztt0x09Vqu/hbaKOqw8TAfnEf2RGlaVfuE8q3WpsS3OM5T9Qe+FVUJN3mo91TV13dqtqGuiFla10tSwnortZekOt91Nfb3SbqsGDy3arsoxO8jc1HPMVnpHvVfs69sjeyL2ojPnfvYh6nGHUdobx+r8w/kvoRS99Bvu43hoZtSSWrcd6nKNfqtauotNOc5PyR79odSwYgxLk3Lsz84b2Ml5eSz6IPrVnIaVpKMb99iUOY/661wcWJaHGkzVKVfPCf6mQs9LDb1oyXFRPo+y/j6Av2U5EdvLO2UZpEzDjCZ56KlLxe8M5KEx7JHoEu2SZZ3S78t5Quc1tmcL3bFWJ+d2aoKbmWdjoXRmeyZoWGUNxt+FUBet641uLI9VTSv7InX7On4HrI0ibsUx/IbVGGOMMcYsGi9YjTHGGGPMopmNo3cI8xY2CrQ7kM0VXiOvV7RbmV4lr/AKegVrID3TCufZSGiU12AIZC3H1uEkSgSmMuWBIYVp+3JXhglu3ylDExruvEQ4kZZY3Wq6ZiUt0DDgnOVWlNZjNKg4JZQEjP1xi6lC2YF6GAKhcrkJnodhmUFtmhgq1+swzIb+qmHByioGkY1SrnHcSouymLaFnEHCPZUlCG67lLDQBk4tr1AHcfyatPY6FXmA3KWfxsWAfl+E1hhKg4xHLWooCQhKgGSsMZRWSFRoiTMwhDjtbxmepZWbjEZKFPIooTTInjg+dpdTnVQyCYb+Cvu4cpfa6Qwr2LpdlFaDWtdNlG10SraYX/thThIgUg7KKlqOEQ3zl/QY4DpvMJxbWk4dtxw6fFk+VjZrx6UblH0Up0TfCHRPvU5X2aMdD9ez/kp7rPLY2mpSr3Eyx7OCASF4nb851gbpQwzdj5Tt9Xv5DEkAjh31WPTbQZ7flACk1XFrxkqShCe61j2bc9QOx/mG8+H6YvoMm1E+F1opb7Pi2u34cnLWqrPqi+9y9Dx+w2qMMcYYYxaNF6zGGGOMMWbReMFqjDHGGGMWzayGdXNx4+g+Kt40HRk1rBvYpKiGdQMdxAWOXa+nItJmSzWsNzal9oJWVRuxY6ksPmj/I5oPpmPsRavS3WVKT5ymmTQyTVve174vtWCN6sSoJ5QyVOnQIHTpi3SgcTaapixXK21F7V+h42mZ+g160uHyic9VyjtsFzYu6FdqSUPrGv7dVlgQgTo1oViWwKqquBWmHaWtjNhn0XaN9lSqKaOVkZ61us0qjaLYwq3OY1e03ZU679120nvR3mQv+/otLKVmbKSol2owZlYyV3Cf9j/2W9qLpRmdGC2QtItVGtZhaoeBcxFO09+V8TAwVSytikRjW+l4JTXiWO7b53JuGvT3AeNxq7Znmh5z5F5TXCb+1kG0f7Ciq7ab4scYBZxTVOvJNKl6LOfsJqgDFQ1zlf7yOMzSXKaVLXdV1lpZ9bfH9/FUiSnH83HNb6Vh1fKcRxZf6OAjkH6Vv5HQ5yPGD8d7kW4Vx9LmatjKuIQ2NuncD53xfl/a/BUaVjQZ1wzFaXGfSfojLQDrZ6NYKtY/mCi3ZZyNeE7m/vjvJ7iO0r5apeSewW9YjTHGGGPMovGC1RhjjDHGLJp5WytYrBRhA2bf0fAiJAErvILW0P6NdRnKvwlLlVs3J1kC5QNr2b554wb2wUprJuReZYmQ1+v7VfkqeythSobrGILV8P3dy8tiX7NDtpsijFHGDbo8nYdSh8TsNhL668+oCdhsyvpv0lTOPUIkSmVVlWDdUYQWaNuDTe2TzKyRp3Zs0Her8J3EbapMKbhopyG6BOmDZgXp6AdDmxnNdHXcfioC4XCcR9u/jjzyWClfe55Q7277eLG91VA/LOR6GTM7jB8OtiLr0DAvCci7af5ZQUqk4TOGP5mFqqhgVDbta7KkRGvRvmqf0w8IYfbledXCh2HKnpIU6SdpXUo+BgmTZ8RuU0KoT+affZ59ZDyj9HPZARkvlbAireia4P3JftQ3rznOhNXL8C0sxijtUEkAba3KI8uoP0P3WgUzWQ55Hs4pOXMulfvEWYvsecy8hntRy7D7Jf96ptjefkuxrf2kHzgJHreNYl1r7TaJWddgzzdM64IMC6xuRmK465k9T8LqkFtwfA/Sd/mc0udA2kGuwiyBIqFqIQ1bbY5vJ65FpEZpcUUbtUH7zVPoKH7DaowxxhhjFo0XrMYYY4wxZtF4wWqMMcYYYxbNrCApQeuZRPdCu41VoYeDPpPaU9FTtdDJ0v5pLWlSb2wuin2qsV11xy2vIiI6Sflapb6Mkr1oOnIDDZnc5xr3lSlxkxPTaof6wiS6u4t1qQdNzaRdqZ2K0BCq3zrjnyMpoR2bqaTrdVlG1QFTM0Y6uR+mlOuRxlXtghJSLHataJcqKSx0a6JPahKFytTGTh9XGC+rlaQ+5UhDIXrpVxx3HCMhlj5jUN+q6VapzYW2XHRG59KwDrBx0VStwxa2LaLtbDNTIULPtVVtJ/RcGKet6KeoJ22lQRvOYw3OI3MKildp/XSzsmOTlLTjvtyXqMeV+x52tM9B35Q+NqKAKct8yMdAS63mdJ79ubyKok5RPRaaTdopHU8fyoeVzpm0GOPY0/FO952sY5Y2a5iptS+11NQGURsp2k9Jm9LKqDr2eHmqdJ0zMkLVdjLbKi269NhMnfGJ2O/KOUWfKXUK2uOWgPVvJPQ3ERjgI8eTajKhdxUd+u07t4t9/H2HrmmomWcb7uW8XF/o+GBqYtpc6WKEFmF5LDX+wyDzLC0e5fm8v88zLNvWyhhjjDHGXEe8YDXGGGOMMYvGC1ZjjDHGGLNo5n1YoQtV30audDvRf1S6sMo7VLfhmQfvsSzCUMi5irSt1EhQu7IX/7OmSqNYCXOKvYpeh1nLqLFcr0VjAr+/ETqmLHq0zQY6qkZ0fpDkVCnZRD/T3kMddSqYKlF1oT28JIu2oXct26ZKDSf7groi9QAsr6m66gRtVe1fORw9doSPnuolG+h4O9HxdB2vCW2iWqsyzSO0TGXqVqZtVR8/nAdjpBN9+Gpd6sNPxTiWHoVJ/XHhbZh0eyj39ZfQwor+tUGfYqrEVvRT6w56QtEsdtB6rTCedH7UuoyoPR170bwxxWInYz9R645t9WGNXVmXI7Rzqjfc99DD9aphRdtTziy/XaikcidkC42uptCuU6HK/eDh1PLHBap1b6hTxvOnV39S+CzLhSrZLM+rGtZKP8r0oVNf4bNJdY303uSxxWkztabHNbep2qcC7Cj3QUuuOlqmCj4Vu8vS23mUObny/9QU8kxRymWAnKennpppXGVMDzxWnhmaivqwXfpLD/rMYF9E+XRNU2mQtY+P9EvF73La5ui+aujINrWnWco+cL7huk79euPJ4zesxhhjjDFm0XjBaowxxhhjFs198uxxPSuvjmdSgjLkzhSvZdo6vDpmyFti4Eyztt+J7Q1CRF2VGlEvglAK7RnUSqmywJJ9eEefR9bJFHJgusMG4RK1gUqwoNF9mSEO2C5pfTFF4SnpkNJN7TrqtHFqj0YbGcZz5XMl5WAZxGaoSmM39RWG+duG9S3hHlpurLAtoX6GU5pWQnsse8P+OtUf7ZQqWxK1c4PVkroisX9SWtA0mob0PHZFbbPHv0i7QOLRS4pDhvXH7Z1ie7g7hdYS5hRKmzpJdbvel/XXiN1TC+uaNeU3InVZr+9jCybh7XGPUP5W75MyF9pcyX5IC7ZDWUf6zdxxnpX2hlyl29AqT8LZ3fnmlO0OfUXG5Qr1rcVqIFsYZ+wZW84/1bNLpW44j0rk+DzkM0XmHLopjTNpNStJmlxzxcd3lTr2eJrUkSlf5dlV2WVJfQ3UhFQeYsevcSru3n6s/ActL9YMxbqFFoVA74wyHsax1YKN6VdVEjBASlC1i6ZonylPRGmdt8P8qJKZan2DfpOlHmhr1VcaEDmW5ZO6rawYK0lKUaAnjd+wGmOMMcaYReMFqzHGGGOMWTResBpjjDHGmEUzq2GlfE83q7RhokmgZpVpO1WNQXsSrqFVt9pCP/O46kagVbl561axrdYh1JFQb7Sa0erqV/fwmNqjfGplNUBPCKenQm+YR6bb1PqjBRN1iVIm+ruckqocogWEBlNT1a3QV/aw5tmrbQp0wFTRqO3MmGFl1Ir+DTq8KuWipm6ExoiaW9UHVXpcrRNqWAdYx0ghxij1Ui10ql0h2IOyqVGNNcchNXjdPT+fkozUrKrv2t4p9w2XkrJ0B51xZTEkdjoY39SidqJnbmENdSGeTkw7GVvoxMTKKm1hkQO942rQeymvGWq5B5ugKo2r2g+NM2MloClrjqdG7GEftdtCA7xRe7bzpPCNqK3piuyXPFjHIeabSluu8zvOxNSsqk2llZFqWDvao6HsOgdSS57RjoX2s5p/dL7hMxhWajLP0jqr+qnAjO5z7PWZAu1rdV5Jgf0UUm4+HS4ffxz/olpKaNQlXfbuErZplfZdfhNRaVbLsbZTK6uRvyuRfaiTDazztL9V2lP0myId637uWU+9MtSxaqPG9PJIFb0TLW+1NtJuwhPhvvWbY6WGPY7fsBpjjDHGmEXjBasxxhhjjFk0XrAaY4wxxphFMytcow5Ldau1Xu/oRqXfbDWNHjQm3N6L2GaAhkyFOD1yiA0N0sqKHqTyh8N2KzqNFTQmqu/Z7cvv7aBh3e00HSP1rcfTlWbUX5Hyk+nuoPks/M/OqGEdqMlUT1q0jZaxkrr00NuI/oZ6vobaHNHJtF1ZHk3BuV6jPNSUiT6S+mvqndU3j1rTUdMKcwwM5fi53KlAD7rFhL6jPr20RWw2U1lb6hapF5ftc2lYoSfV7RHpVzXdIaW67Zr6XNF+7cpK2aCT3RK/3jX0UzfmEgVizOZh0qImeIZSb3Yh2kimeO3l2B2uP1S5i0VPyrTBI+cCuQ5SHA+idR8ruRn6n9z3wNzQJ4QS4lRoz4/r+5rKL5XHTtsr1Av1rzr/BLWxeh48J1q022p1XMNKH1b1rR6gS85FH6SX9/GU43kux2ZE7ESrSImjzqu1nzRT/h7XzZ6Knp7MhRf58fal32yHfpOlTQfc9w5rkUG1ndX6or/356i12I2sW1jX/C3IXjxTmTZ4LH4Tg+LsoCeV503DvMwzmmUqT3vpt1W6+5F9Ve7lKfjF+w2rMcYYY4xZNF6wGmOMMcaYRXMfSQBCIGr5UdkYyStohk0RjhglzJoRch0oNZD32T1tZeQ627EMyd3tbxfb6830qp1v7Jk6tJP7vljPvBLHa+6727IMl1Le7R7hzpGSALVkOp7Gs0HotvqLQ47NZ/x7pEda2n7UcDj6ihw6IETMSELRzeAFNsLia5SwSNtA5rGaTry+gOUQs8hJIZjGlZkIh0EscjAmNAzXIO7ajmXoZWgklAa5QM/YqNqQYLzordBeLiBZSHI0+9WpWCMNaOyne+tahI3k2DWkQiuE7zQn7RiwNIM1SyP1t1mV572QNmwgKxkh69mL5IdjrU73PH2XoVItT2JqVqYYFmu3jL5ZpaOWEHqlBtHx0ZT9i2muk6Z/PlOYN6K2XtLbpaxLpVrzqU7L+k+UVHH+kXZkuklNBb2CrVWDlNlFKm5a/DBtrsylbDe9T0rH2DS9WnJREkD7yF7nFEhNBk0tyhSbkOIV8hHaEJ6G3d3S1krLQJuylaQQ53OWc2AvEqXKUBH3pnNDhv1ZrzZ2lLXRlVAlP5h/ON71y0yZrEeyHWglqvey3/JOj8uOdljTDNIBW9xYy/WijKVxJr0v8RtWY4wxxhizaLxgNcYYY4wxi8YLVmOMMcYYs2jmU7NCIzOqtg/71KqK2hDIXmKUdXKGboR6R9Uj7UZqTkTbie9tkcpxX6SDPW4/dLjOpMXYZ6aE1O+VepS7O1pXqQUENENMO1jUGbQrRdmP2/lERGTRiuT85LUhT5cc0MXINttUNTXUelEkmkNtUqh3hR5IipBpuqH6PvT6tCrLp9ohWgVVti56mYbaIEl3WMp9Kt30KPouYUEAAAAgAElEQVQkOITF9v9v71ybHEWyJBoEAUiZWdUzu/v//+P2VGWK935os8HviRbdbduSycbcPyUlhFC8oHQP7rRA0/ZjBGiR9gKQV7po9ab8GfvvUWpz7MNBT2lg3x/nNMB+iEzrOh79VFnGrbQGEtsygJJqSZNpPwU2TaNbt/U+655SSrP074a5rzxmhXPhOON48HAzxn/7NoTtrpeYWYx5jfddYInTtZhXMh8KGeQHaicTvt3nVIP1Eu2esN3K2oCpX1mFKRvYFc4naRdMH1rlRW6V/C0s0dSijQyrvHev+ML7H8nTqSwCm/tcry5yfN+CuTWLvRujbB+lrx//G7ZXuU/o+zgnNM6Y/O2NbLmMPz5HsBIYDutI3HdRm8w/slGTw/AaQeXAjOJ0tvvXzR3r4zzrtQfrIZ4F0bVz5XM5Ojbp7EVbQrkW8Xp3Jv/CalmWZVmWZb20fMNqWZZlWZZlvbT+wNYKNjhabsa+seTNEivTrNTSINY81o2lNUmmYHKPfE5lh4WfmbU8XhqWr6P0p+6veQyvDVL7qSyOcNwsaTy0umiZxiLHWmBts0ib5Cq9CmUh3fwLdhH/f6FNxcYJbkCh1Mqyx7pGS6JNbK92lFILSt67lC8a1Og2tdlg+kmPsvqgeAvOj8kzIaEsHldLvURLRoyHL7F3mlEjGYkTSBoYrasaqQ3tsAFbiVDIeM3bc5CA68BEuuOcyo4yv3wXztkepTUNnVtvwJV6pLcEFIKpfLJRWawB6xErvYUJUKyJybE2rp76Gsp1tynOh9t4+/ffU4oDowdS0TfyvWmzJZtv19g+mc0l44bWY49UnaqkSNr99q6RACb76N9xThR8ZpFyeOUMpusyrcAYUKYld9ICRE3kWsUxp8NqZkkW8z3L9XFBTXa6xXG1yCIzAadbZL0hFjMj7XGRkvuzrj6fP34N27vagvWX8No2aJIUbn8qHzX5Bn/ly6DPAo2Gvqf9k17Pib10TDLT6x0G5yIDpeBDeQ+xCsax0xKOFmzSRtX9j14LT9LyUoqWWFWa34n8C6tlWZZlWZb10vINq2VZlmVZlvXS8g2rZVmWZVmW9dI6ZVhpoaQWOowl3UJ8KLgbWlcJQ8G4Q2IkGpPadJFHCVgEuIyOeZsCXMzgeTba1wgQsoFZbISbJHtEOy9ld6tEQ5yv8iD7RnhOrCTIrJLzau6zxI+Uck4ppTQJkzlNZCeFvWpi+97GyAyv27HdgsGsRu+uPCn6QtlO8I8bo+qEf23hF9KC6Rklyo79pnzuzPeB8ZlkME9gmL9ukVXbd/GCYrSfsJSM8tthIdarpw8hqAfp/QIOVGySZozlURNBweftsGLaNdoWbHODPlQbu5H2csKMMup0Bk/6eTt40mUhhwiOUtcxYmvy2jzHzxixPUtI5ArLmZVRxfsxbho+VyBc7+WCyFH0kdoateVZZGKq1kiNjKVtlHLxaJbfsbWSj6BtIS5ArWSjZi4/Avu2vHBhXPUXAYPJsGKchdhKHletvficBi2JhP9v8aH7EseVMocN54SsTdXzFVj3lWPkmHuUbj9jNGvWHp5gGbZej79hjUfbP11cefnucM3Q+c6YVLUpawk37/efKyh4DqPr8JmLXl84V46/KxYcpzDux1igFVnPrOKk/Yt7ERlTO9bDFWun2oTZ1sqyLMuyLMv6j5FvWC3LsizLsqyXlm9YLcuyLMuyrJfWKcPKSNXAgvFWV9iLhh6j5D2UaSWHs5HLEi/Gln6u+Xf//r3PjLgrzg8n0RTheeAhq5Gq5CT3Ex5lq/5rQP/U47i5J3MinN8SmZwGQEoWPnN7TjJeSimlFfyuok0LASDhVzIYmRtYxVlYGI0dTSmlHfDLIFGJPO4uTPNOj1aOFdkuXex/sonasfuMCSNjsh3wGeKnmVL05lwxuSZMROWUC/YdZ4keBPNLxi0Jv3fpnsMmvl1iO0zBQBUMpqQqzhW7jbGgnoTI2+TXnqfjH8avn+G1TsYxObURfPXX17E9gdGqkHXlMas4ZX0fYluxrXxuO2D5pn+qRKzS+1NZ1IKx2cMrt5X50vXPYZ1/E1hAuW5w3VPWvPaAjEcNzyHgGQX6gu+NeobTXFXG3IbnNDauY/KZWEPofRk6C6+pbyxnbMn3GfoG46hnlmyYhsx4lTWFbctdTzyOH6Xp8zNst8qB0odVrv0t1rzmhJkmL9zxfkPanv0wqPcrY5nBjCqaXbD+kKnXsUm0fJG1v+Kg0S+6BDcYby3Wqnk61jyO20a9Vae4Vi4jGdbj/Ph8x5n8C6tlWZZlWZb10vINq2VZlmVZlvXSOkUCWKkIPyXTm0V/ai+0w4q1qk7sqWgBwVKKfiZttrS0z9JtZR0jdbeMMj/jYFs5h9KjFCk2MzssZ3qU8rOWkHjuOMFVLCFowxI8sYAzzLTTufP3o7Xj/z4a1UtcQLuYJRLGuE7CE6wspbEOIocqiN3bZQxumdGssWyUZUxmjCvaF2kEbHtiM0Sbj6YMYbsMx/bXHPfVCNCUYnQjPXwUm2hmWi3F8Zrk3BlB+yi9v8fv3XxK7CO/t1RrOS4YETlJG+UeYxHzW0v5N5YT5bBcQr6+Isah8YLjLb62AOMI5VHaLkk/DNdorZNRttQI0AtLhLC16tvjWBlzpZFydou41dIiAlnO4TI8p8ybUkop3788Vcu79H8LS8MBiMggSEAmxrVzTRFrqEKUTK37YgmU1ypF3TaU1WkPtAL7Cqej2AxjNLGvWr81tFxMXGNkG5+v5d0Zsa3rjj5S68snXYAWzL2203GP2GFpM+IgO9qouagdVfzMFn2oFpY9xt+brO19ZZUHxE+YgIz7qLbC047vMvIeRq4LK2+GeHvRH+1FbKxgW+PGd4yTVuZSwWfwvm6TtX5GHPuZ/AurZVmWZVmW9dLyDatlWZZlWZb10vINq2VZlmVZlvXS+gOGFfyCYgiwblC+omItKtso4T0ukWljFKZafDSZ3iz3P5OMaGDgYBdB3kzfmWkXITzPBm6p5+2/cC5k7oiVLMKrLGSIdBufQdubVb7Lk7DE3z63sjc5/ibLNI7CViGalf47yj8vYFi3ERYc/dGGK6Lq5lXigBtwTR14UmGQChioJoMbE+gxV5Y4x/jIcxwrOL00C5um5/rbNtnoY7vhZ0oT0YKG0ui/Z/HOHeAmTUPsFnDd0g6coyvZQ02+xJdZMADHfMynH+tXeG2XGOEFvN6PH9ECS9n3iZHCOF89BdoavV2PuMgtDsXUgYXsJZ5xg40Vdk27zK2ujztfr8fOV1iNDVes7XJc2pI9UrQ4VN6v7cCzS5vikYmU4Smo9nMN+N1mJfetz1BEvniV8UG+PufIxWv+JK2CMp5LWOUcaHukz000ZPpxnd2Esea1fL4hAvh2fJfxK47ledToTlqy4RmURdex56wqZFFLf0yiltZL2p4bn62gd5VEq/N+As8V6GVhI+ssc5qR8V1lISevZz4XEqVR4IxoV0u+Bd9rgW3ZIPdRK5+JWcBmyxgr9MrTNY73Nzh3fXSJvOuZ/AurZVmWZVmW9dLyDatlWZZlWZb10vINq2VZlmVZlvXSOmVYq/tZYRYyY8OEvWjIt8JPTzmNDvzgBUyrRqwyUlW51cpvDZ+pOM3GaDJ6ZAobRKZ2E550gw9rywxA8fAjl0qOLXg6jvEzld9bwCXy3DfhU7YnMUQppTpuMkTYghOTcdQibpd5sovwhmS2ZnChk0RurmCtNoFGNwKkmAalHPxZ7f0b36lMJn38ssyDbYsUzzhFjuhf/xJv0PG+b+1vxxWuG3NU44tbRhBXfZSeLrKHvUbbJvoMHn9X/sz4LsrjclcgwSmPwu5e44tfwmz9nH6E137MkXdVTnVlZCXG/BJiMqM25SSxbnXwZS3fZO18w76ITS3iC93Fw6RB2Lnrhcdh7OmxPVSg/uM0g01st2O9rTyZdVzN8TWyf5tAzvRyXsH7FWlTem9qR7ZYJ/jsg8YO06ecEcCz8KbLDm/d5j63ODFmVnhN8sCZzK20UgvvTY1xJQ++ZTL0x3Z+khHrpcf9hX4sroHL7ZjffB6hYCzsRdqTVqZoI42zXbHzKvcmHIsFvKueO9drPmkwS/9mxFoXGY+Zfr14uGWUcTMxQhX3OJscixG+ygAvjCnHNy8yB8kZn8m/sFqWZVmWZVkvLd+wWpZlWZZlWS+tUyRghjWLJvbRwkfLvLQCOttm/Fjp+7v7DsPl7r47zodWIWq/wajQaYo/e+td/CkSsER8IeNn70aRgDnGx7HcpSVOWmC10gaMkW0yI/aOkgetdR4qli9ku0c9smgpZo+oxDSztHF8d5a/M/pGoxL3DdGIa/u7f6eU0ko7JdneUVJikUstphjbukjZf/yKffH1A4iAvL7tiDLG/OkFWSgt59bR1kQC+P9T/S6cL49SgefUktXiDOuNdGHXE2dASU5rsEQhUDKcZZwsxFWk5P1jjrGt9CIrsuYUeClNiFxsZd4OsOAbhqPPLv98C6+9/89H2P74dvT9MKDvMR96yZPsaV0lSEBbiACEzVS0Trnfjw39u1XNNY1MbohxbfIaSqC0ypMjs1RO5kctqFhKVd/AKlB8wRhUiyKu70QwxLps2rFW6dhl/O8M7EwjVbE2TbBsUwu3ndcNve6j3M1rleJWjOt+lC6YT+qxOCJOOUlkdy7x3Gm5pyOwpcUUkABdrLjvKHZ4BaX7HQjkPB59yLh5ciZfX8f6NN5gPyVrE+Nfd5Tgb7fj9c8f/4rnAyRAu3/GfdMuA4V2bET9kqz7K9C+M/kXVsuyLMuyLOul5RtWy7Isy7Is66XlG1bLsizLsizrpXXKsNL2phW4iVyi2v8UvEbLj07Y0+vlGl4b+sh09MKtct9et+k3RNZGsI0JdgxtgrVEVoYRrJQwbg3jYME4bevBeMxgXuo4PmWwyKMoH4x2z2Bi9uO9ZCofqYo3U3secMo5H9+hSWS9cCSxqJkRh5ca9tvxfecRtmFyCvMEhhk2UuoqRi6ZfKROkmmMffHrr9Pv/p1SShPOoW2Pcd4gSrLvI9c4yPxqW0QbS7vvYC5pmRTmZWWh8hhlRPE2aZbX4r692swMaHewh6vwm3mIc2QAU6ZzOHfxQ4d3sfAp8Vzfxve47+X4nAxLnK9bZOfU8mwAk6yc5PtH7Ot//vcvYft6VUYZXC/YtE5e73B+wcKnYjO5Vh1/N2T3HijipTHumeescx+vLWBEJSiyxXozIMJ2llzNjTiftgvm1pzjzp1e1/Ccxoy5F2whafMX7A8RBwxLolVep8Uh993VlmmCdaIsuxueDeAlZpQxuDfwUnuQWlhD6TMp00gbyKNNeF/S94xEP8YJOWjOvcugz9PE85vGYy24fSEi+T2uKbpGV88V4Lg/fx5s7HjjtfHYeUKEPO9pviSm9+tn5PbHMVr56ZV+Jacqc3IYaN0Wx1+RB6Ia2oGeyL+wWpZlWZZlWS8t37BalmVZlmVZL63zpCtaV8lP1BllNv3JvC493re1qiyv8DP92b59kTILy34pKisGgJ+gG6YZNYoExLKgWqbQCGhHmtEm9kj82ZvJPfqZtMXQ0sAOnIEWJJo+QTuLR6rCRxQDgMVLbo5+ZDpUamMZaZE2rZKaEspli5ReYI/VSelvRNpSScAqxpO+aO+XRCekV33+OM7vdoN90t5j+3hvD+zjCjs3tSzhnFDrpS3R6g3jXP6/utJm7VHCPGgDfhFfUwudocNalDBOpHlZ5m+BCBSxdCqwFLoIElDeUKqFbdTbt6u8Fl6qkACtE+4oUWtdtUe5+Pv3aGvVScmapb0J1jY6VLsW30VQlxUpfPiaqQn2Wc+zytuxijfSyLSuas5KlWv8fpvM9x3zGc2UZmmMjcxKSCUC6kJbRVmntyqdENcNWUxpD6RIAO0YtfScUlz/Z4w5WlbqOsxEQZ2HXINps7jJ9saL8INEe0O9tG5VouTRZg0sm2jhpO3HMCZeFzpZn5jWue06v+Ma/PYR+6xI8meDz2Bw5afYZTGhag/vjSe/wNJslPfOGEMcU5rwxWuGzsG3La5j70C6ipzf0P95dMS/sFqWZVmWZVkvLd+wWpZlWZZlWS8t37BalmVZlmVZL60/sLUC9yJMRwerok6trE7ssFKKLBKJQLIiil+0YPACN9vQJgo8nHxQqSwqyOoe8M3GMwybiACs4tvUOgaMLc5X2d3KBqxIHN98bgGxSQMuyxN5M1rqKHvc0IJGuDCCOYgiHLqD3ySHtfC9qzJuYK3m47Wfv0amrVkRpdcebVyPDYwdsVQhRzR+HsfZZ0CO7EbZJpfaAAbTuOAGDHAWXlxZ4d/eB95V2LRnpfg2ZCCF7yLn3Sg7h34osGkKEZr4moy+LMK0DpfYnz9/HH9f3+Nrl/fIEn/8clhQ5T7ue/uKPKmO8/ET9kNiK8O5/47P1LGxgoX8yuDZxeom0y5OIlbJbdYR07KxPe83Dl5/1Kono512GTu08yPrrgxrA052xhzWMcnrmF5zalwzHkc51eaC84F1kMZ2b2T8ZZ2dEGm54JkFbQfO73Ul0yrsKa5Vap3XbvctIFOKiPq8PoeLZ1xshIt5zTjOacZzDrz3yFn6BdclcrO9jA3eM+yyTb51+AHuXO6jyN/yeqc8M9cCjaJvOcGhTQYH+VYeV3/iZDTwqgww2n3ooiVpkmtR158/SnXn4y3LsizLsizr9eQbVsuyLMuyLOul5RtWy7Isy7Is66V1Cg9MYGKKcKvkPzphK9cO3AO82jqNcQXLSR4lJ/UgxQkK20nWkPyTsp0FXBA91ZSBOfWdI0JJCEZeJ5t2wWfq+U4TffmOv0f0CY+rnC/j5B6pAr5L25uxhfum7RvHSlfoUSn7dgP2jbBiJx6u28w4uqMfF/TTF3xYlUWumMsKKROmDH0z65wgi4appz2l55pSSlUibVHeObaBsktdz4hk7CtMVOXn+jBxjgjbh7jNHHxk42uF7JxwWgU8aYeYwDX4u8bvncU/cCVP/Q5vwe/Hdn+Jx1k+EL8psb0/6Yn6dryX603fYX53yrCCNdziOdzEf7TguLpstGBfK49TmaPPjGadwc+tRXyDmdsqzyzMmPsd1xiJcKYl7r7Ts/L4mz68uvZWFCWOo/zrZQZDn8gmypzAdU15w42MKPpYPVHZXsvJ8wAbY5plPSRTy27Q7Wl+DsN6vUTOe77JWEDMrHKhI8bXzGc+xAOczG/BMzxX8Spe0CjK8nZj7Pv8M/qcZjnuxvjpyvNW73/wjIys59t2vna2cs1Y5vvsdUopbWf+zbpu0LKY64/4SQ8XOtrfl39htSzLsizLsl5avmG1LMuyLMuyXlqnSAB/2tZfzGmxoJGgtFOiO0ko+/Nnbhw3xMShzKIxpRvKPlWRP1h8nJe1NvmcnHg+x9+09BjxvbVNaMnF8qeWs3uUvhW/oF0Ty/5azs75z//U/v9Vg/Moar2Ec55GsXtKjAslriGvoUQ7o7SmdmV7YbtovCBKNrCy2TY5P1gt8TOV+9gxrrKUAdmnG62q+mNflu6raGOxmSkFx22O9uwQ8co+YsnpGWqrSE2Za7Be0tRMVilZnt103KPvOyACMY0TyI9ECLIkPcBKq5fz7THVMuMt5b37QIszsWvCejMM9y33JqwhSKtNSgjwuLnRcXse4awlw6byY3ucPomkyXrW4bv20o/EpKhF5vQGm8DC2r5a9wDVaotiT1iXGeksfTX/ZGzv/ZjubSNmpOfAz4zbev1mKZ8l7kVePytpI2E63eZ43K9J0DbyFg/S+1u0TFpnOQes7WqhlDG/aRG4ynVgAn5BrEy7guXvUcrqK/qMFpvKf1WubkQXpb953BDhi7I+740CeIXr27oSCZA1D+NtuB6TkhZwBXjaIBjH+7ujWS3LsizLsqz/EPmG1bIsy7Isy3pp+YbVsizLsizLemmdwj4ZGYeNbGdydkX5BVj2kKVs7m7UWa0KciAKbNk1ehD33gBAdo27A3+y47i7Mo6V/ZTsS/wExxmFeSLHyS+qTNEOliZgvGCPKusLPU56nsjbqM1GbYUjDbffZ01TitwaeZod+24SG9kwRlE+Rtmk396HcSUcD20+yOYEoYt7iZXtAN2RYU3KCYJ3bisWVV9nxO+xb5MYJYm2DtZGz+Gda9sUmcNokyzfs/TnTNQotkY7WVg0ddepjUscU71AjGR+OzRRUf4WYzOv4C/1b3C8Gh1aTtjH33YWixzYUW3Y1sTDBhO02dSqChxntZZqfO55zOPfqRuY0U76leyfMs07WPeVfaHfgS56nJYyqTcwj1muKf0ACzleHHblUhlBfH+lXnnNk22OT66zwaqKwwjr7iw7jPhMwVLThPb5hAXfz5s8y/KksfLx8R62l1mtBuNrs/TniDaZlmiHNq3HWsXlmvGr2ma5wGJT7oeWE+uxlFK4hrB/6whaPQ6en9FNHIex9bOswS2+F7+n3sd0GPNv70dU9eUt3h8O4IyH67HvFfHTZ/IvrJZlWZZlWdZLyzeslmVZlmVZ1kvLN6yWZVmWZVnWS+vcsI6MTFK+K761k8jAUjGs5BTFCw3sHFk69WWlv+uWxZ90hl8hI/aEY1rhvZcLmcHjWIzmVFaEfnU8rrIiZDwZHasxduRaQrvj/xjkEqOH3/MoVvqn7ievXYaDWVnAhO7kVCUej2Oj5PuxpA38U5Nwg2yVkqOXqcbGdfA5ZQSeeu2ujBbNB+PTlcjpkFPWWMCMuZXpK9n8Oc9Wvo/8rfbLsxjWBqxVVp4PXJ2efg+AVP0UU0pJMcwN/w3HrsGnda98ONVbE+/bEDUpwOOeyMXT41GPE19TLp64N+MPdfQWsHIt3qyRqzxuYIfBsPLc11mP8zxv5xF9I+hxahnx2x7bAxhmfL0kXyfM35SiP21KMcaSS0or84cpqRxX6tNaMazkIzV+lV7oJ9eUKkZcr5049x3XEY2OnjEPF2nPCT68n4B+R/Fe3U6Yy79Tyk6mFMfNhPVmlDa7jLHTxhn7TsKwcn4nfu9j3zYh5lrWVnp1b+DZizzrwOvUxudywuM9fC5D9qvGG6KhdYzjMwrXXYmK7ntwqpfjWnl5i9e7KxjW67sZVsuyLMuyLOs/UL5htSzLsizLsl5ap0hAVebVqDfabUjNYe/wszfKLmol08OyJ7PkHSyQ7p/fjvgx1sB2+Y18Q22nbYglBG+JdE8bSmfLwlKalm+AAOC9k5YfUF8KqEHtTxI29ef+fGbB9DercueQ7W6IZfVOLGhutxhTOE9xW201NkT9odIR8IjKREzKXIy3LcN9zKIuw2C7qHUMx6D0RRPLJyujOxuNnUR7wcKpFYSFlle6XfAaG0XbgYjCw8ROk/HMkmvS0i7tnTaW8o+/V5bvGDeo7yWCotu0O4M9UiPbtLKZsBbo2rmi9KhrE8cXy3eKpGz4zHmMtjxqIdai83Ws1kgASsvh9XOK7O/U5xcjTI85tIP72AQJWNrYvlwFNaabloZEydqTUn6RknvLmnuVJazbFR8WNsO1lWX+sNDScvE+ZjZVcem4tguGtMFWb5H+v8FWcUQ/jDJ2GAf7KH18/x62ZznHmRZd8hrtu7DCpF36aQQ+gGYI+F8DvErxAWIc7N8Qv4rmI46m42gdMYdlMW3h68fr394qAhk/YwAScBErK77WSzTrx/eP8No//uuXsP3t+7d///3+Ea93Z/IvrJZlWZZlWdZLyzeslmVZlmVZ1kvLN6yWZVmWZVnWS+ucYa34TY20I6d6MBMr+S3yruPBVq0FbB/uoedGWCswHBppN4JvJO6xCuPKaLINrNQk51fzKIcW8G/zFBk3jdHbwcNx3/F28GdkbNcTmxOifcpCVpG4DxQtkwRTrWNzhREln0mSaJ2PMTet5Ptg0xSY0fvjgaxXBbwKt0aOm2qVAyWfKR+zVXZkYBMlerQttBKJbVTUQq5DfLKwaA2ia6vUY/mb0X6P0oK4TbUVYrcog7mjvSZwqTqmNtoatSN2PfZdMaZCVOtJbGJKKU0y/9lntxERtDKHuT7qF6/XuKi5guAO8bjxu8C+SRaOvWJY43Eb4RK7k8//uzUiNleHQCaZKiz5DEupdgdbvt1nWDPHTtJ+Q/yurDFq9/N722p7xnlJBb4UHKhyoRyeU2XLJPZOGBtkFbVxgTCnVVaKaY/fa2IcuT5u8aSh8vHtW9i+CW96m+L31vuE73RGow2gPCvw+RXnM9cN3eR9k9pxrpiHDeZ7fzksnrgmk2Ft5SJ7ucRrhN5f1LZWiJwWFjXjKqHPm6SU0lU51fdoVfVNrKr+8Y/YJ99/iZzxt+9HZO7HR7QlO5N/YbUsy7Isy7JeWr5htSzLsizLsl5af8mjZNu1NM0EKLW8OrHpSLF0NX7Fct2GskbXHfvyOMF+CrXxBj9tq0XFenI+KcWf7TOQhSwWETzOgpKRIgHEEBaUu2Ypf1b7ynFZ7qrSv6QfaqTjccooc2UpJdACRC03MkoOpYsWF1q+QLOkZWHNSdsCqT+CLLAsMzOVSAqxBWkeVclWS6Qn5VImDVUeKnJcYigN0qy0jdrCKSw4AzANjledT3sFDDxGbHvtp33nvJS1AUOZFk5qP9ZgTDWwddE+pfXOdBMMgeO2GlPy3hHJN1MsISoCsmG9UXs82mOxW3Tfao07Sd6jBZ8mODVI2+EXbxW3WJ/3G8eK76NlbZbuF/mueY+4SEE6U1qlb8A/cB1TJIDjU68/LY9T2UidYB8VunNiqyhfeyYCsnBd22Rf2vORJRNMApNt1TUYC9cMW6tp0fmcnqL3t/ew/fl29O/nLY6Fq7TRUq3BSM8TROD9G+yxgDbNYV4SCZB5iDbh+q341/wHVp2dnN+G/pwVu4QdX4frwiDrYcvrOKb7ZTj2/UDC2LukWb0h2ertLe779n7YXr2/GwmwLMuyLMuy/kPkG1bLsizLshpuLCYAAAVmSURBVCzrpeUbVsuyLMuyLOuldcqw7lX02/E3OUtlOnKKfBldPBrhxlZakICvKMJpFNj75GA/FBmOUmABIVwgI1TJkwaGtY0smvJGjO2sLHKkjWhjtcJqQttzWeK+t+lozwnnSruSaKGRnqamgjT1xdgX+t3X/T4/mlJKu8atwnYkl8i7qn0HGa1NLbCACWYwfPF0eT6wOdu1vWl7cxyIXNhG2y05LpnVjIjVYJGFMadzglZqVHjn03jn+1G3PF9l4dXeLKWaBQs8H/4bTk56l37iHJlGfY3+TgkSLhGvLThw0XWtYk1lvu+YKzxBeSt5M3Kqu26jbZUP7XrMK8ZF6oeSfXygGF+8h7WOkZtiRQiGtQXD2qjNFT28drKnug2GVcZVwRir2lC3q9RWjM8QmYzTC9dgPEOBPlabK/LAVRSzrIENMWrZlazkguPo5ZyWTY9Sjyjrt+vBT76/x+u3QpmZ/CjsLYfLcVxGvPJZklmuy/VKKv3JWPDqGRnhUqs+47Xy/jU3cPFYJ7huBBabDCu+TC92bdfL/Xb/9i1Gs358xO3L5di37y/pz8q/sFqWZVmWZVkvLd+wWpZlWZZlWS8t37BalmVZlmVZL61zhrVCJO5HGqrv4Ab2q+IJhbMchshBZPA86lnHGMosXBh5DjJF6m1Z+6fSy1Q81cBwaNwpIz7JnGh0K31rTxlWRL7exvsM68KYWfmbbNQjte/kBGXjLKOP0YPk+1bdlawXYVR9He2t0Xlgq2C/F3izZYGfJvq8CF/aVDGK4aDxONW40kjVeELbxvcKAx4PE8ckGSjsGwcLX3yMyGEpT1r5fwZGPY77lmNbmoycXabprc5/+iIm9aPEcdCHe2BYwSy29IW+7xHcyb6MFN7xvSPzC345Re06OnBctZClXzPXDY3x/INU0b9V9LUObUO0VxhW/TullJrKh/VgXBsyrJXu+7BqPxY8e1H5UqvPMi8qZxHozf31sfITZ7Rx8P888XpNMQ68yWS3j8lVPdfSYK3SsfIkI9brNXp+hkh0rPWXr69//81IUMYpj/LcyTjd91hPCf7SJ1+bcemZcbD5frR2xY/LNr199Xw4bnlYXQv4HEEHzrcIw3oZ4rMVkR2Obct9B9m+wOv8TP6F1bIsy7Isy3pp+YbVsizLsizLemmdIgELagxte9/GJdZc4/uaEZZOYrcx4jXaWjXB1oi4gOyL8gNLMkn3pc1SFdUqFhUsUxZFAuJH0CJFba+qZE7ag0jtm7iAYgDTjLZk3KZ8Jm1hniktXdb2QGINhjFWx2GqjRgjNzF8Q7n0fl/Usaiw+VDUIMdSRtWmGk3YxNKGlr9ZvqWdieIOKxCAnZHEaoGF0t4kY7ctnEu0QRIrmyfVenuUf2aNZgWaE+Y+Sl4saaZgIUY7PsZQ/v77UkpJ3fDWjZ+JdU1OocYF4nu3E1SjCfvS7uy+JRzHccs2anXuMFZULZkw/hmXLGXBHqW9R2oFjjPLd59R5td91znaKqYdEZe6TjPSm8NK+qN2fjtBQiocS47Dw1RjRzfwmowjlqWpaGXF43B86q6VL5y8dj++9Le3HvsWrs8P0pXRnrKWdbBe+i5lfqJ3M3A7RQIYg1s1vVqRVeuE2kZhTUa/KOZIrJEXR8UMubbrOfDaU2GOuo3LW8E1JFi54bVermkD1omuA1oga8pfcVT0L6yWZVmWZVnWS8s3rJZlWZZlWdZLyzeslmVZlmVZ1kuroTWTZVmWZVmWZb2S/AurZVmWZVmW9dLyDatlWZZlWZb10vINq2VZlmVZlvXS8g2rZVmWZVmW9dLyDatlWZZlWZb10vINq2VZlmVZlvXS+j9hnSiJ9xk0vQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x648 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualize the learned weights for each class\n",
    "w = best_softmax.W[:-1,:] # strip out the bias\n",
    "w = w.reshape(32, 32, 3, 10)\n",
    "\n",
    "w_min, w_max = np.min(w), np.max(w)\n",
    "\n",
    "classes = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']\n",
    "for i in range(10):\n",
    "  plt.subplot(2, 5, i + 1)\n",
    "  \n",
    "  # Rescale the weights to be between 0 and 255\n",
    "  wimg = 255.0 * (w[:, :, :, i].squeeze() - w_min) / (w_max - w_min)\n",
    "  plt.imshow(wimg.astype('uint8'))\n",
    "  plt.axis('off')\n",
    "  plt.title(classes[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3.7.6 64-bit ('base': conda)",
   "language": "python",
   "name": "python37664bitbaseconda92b0ec200685491790e4a861efae1222"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
